Is there a way to run the unit tests in a debug build and performance tests in a release build without manually selecting and running individual schemes?
I have a unit test and a performance test scheme. In the test configuration for the unit test scheme, I selected debug build, and for the performance test scheme I selected release build. If I run each scheme individually, I get a debug build and a release build respectively.
If I create another scheme that runs both of these schemes, then that new scheme will have its own build configuration. If I set a build configuration of debug for this new scheme, then I will get a debug build for my performance tests as well.
You could use different test bundles and include/exclude what you want.
What I mean is creating your own custom test targets (bundles) and using the Xcode Test Navigator.
So for example creating a MyUnitBundleTests target and a MyPerformanceBundleTests target. They would be two separate test 'bundles' where you choose to included/exclude classes, methods etc.
The Test Navigator displays a hierarchical list of the test bundles and associated, classes, and methods etc included in a project.
You can enable and disable test bundles, classes, and methods selectively by Control-clicking the items in the test navigator list and choosing Enable or Disable from the shortcut menu, thereby enabling or disabling the items in the scheme.
Testing with Xcode - Quick Start
Testing with Xcode - Running Tests and Viewing Results
There are several additional interactive ways to run tests. Xcode runs tests based on what test targets are included and enabled in a scheme. The test navigator allows you to directly control which test targets, classes, and methods are included, enabled, or disabled in a scheme without having to use the scheme editor.
Not sure if this is what you're wanting but this is the only option I can think of for testing without going up to the Scheme Editor.
There might be a way to achieve the desired behaviour but it is more of a hack.
The main problem I see is the desired build configuration for the test action of the scheme. Since the build configuration applies to all targets in the scheme you need to find a way to trick Xcode into building your target with both Debug and Release configuration. That being said, here is the idea:
Export the release configuration for your target into an xcconfig file (xcodebuild -scheme "schemeName" -showBuildSettings >> release.xcconfig and make sure it only contains the release settings)
Create another target for your performance tests that builds the same source files
Set the debug configuration of this new target to the aforementioned xcconfig file of the release configuration (that is basically the part where we fool Xcode)
Add the original and the newly created target to the build action of your scheme and add the test bundles (assuming they reside in a separate test bundle)
Select the debug build configuration for the test action.
If you test your scheme it should now build the sources in both debug and release and should run your unit tests on the debug build configuration and the performance tests on the release configuration.
The setup is a bit fragile as you need to add new source files to both targets. You should be able to automate the process using a ruby script and the xcodeproj gem though.
Hope that helps.
Related
When I run tests in Xcode, as long as I run all the tests collectively they execute normally. However, if I run only one individual test class or one individual test function within a test class, it doesn't execute any tests at all and prints:
Executed 0 tests, with 0 failures (0 unexpected) in 0.000 (0.001) seconds
Upon debugging I've found that it does execute individual tests if I choose a particular other scheme I have, but my other two schemes don't execute the individual tests. I also noticed that if I edit one of the two problematic schemes to use the same Build Configuration as the working scheme, the individual tests do work (each of my schemes uses a different build configuration).
How do I make the problematic build configurations work to let me execute individual tests?
I was able to figure it out!
If I click on my project in the Project navigator, then click on my project in the left sidebar of the page that appears, and then go to the Info tab, I can see at the bottom of the Configurations section it says:
"Use ConfigurationName for command-line builds"
If I click the dropdown and change it to a different build configuration, it makes the scheme that uses that build configuration able to execute individual tests! And, as one might expect, it also makes the scheme previously able to execute individual tests no longer able to.
It would be nice if I could dynamically change the build configuration for command-line builds based on which scheme is active in Xcode, but as of yet I have not figured out how to do so.
So for now, just know that you can manually change this build configuration value whenever you want to run individual tests for a scheme.
We have the following project structure:
Workspace:
- app project
- cocoapods project
App scheme:
- app target (run)
- ui test target (test)
- unit test target (test)
After upgrading to Xcode 10, Xcode insists on building the entire project, including pods and ui tests, every time I run a single unit test, which is a quite heavy and slow process.
If i create a seperate scheme which only include the unit and/or ui tests, it only rebuilds the tests when I run them. Just as I want.
However, as they are no longer member of the main app scheme, I can no longer press Test on the main scheme, as it no longer contains test targets.
My question is then: Is it normal or recommended to have seperate schemes for tests, or can I prevent the entire project from being rebuild in another way, when running a single unit test?
(note: I have set the Host Application setting to None on the unit tests, so I don't get why it always builds the entire project anyway?)
Why not both? In your "Test All the Things" scheme, add each test target. This scheme should be shared.
But when I'm working in one target, I make a scheme for it alone. (More accurately, AppCode creates one for me.) Such schemes are not shared.
I am having one solution with multiple projects.
I am basically creating building blocks to build more complicated projects later on. These projects are sometimes dependent on one another, thought I am trying to keep it losely coupled.
It is sometimes necessary to debug one of the projects to test whether certain functions actually behave the way they should. ( I am going to create test projects for each individual project in the future. ).
Since they are all .lib files, everytime I make a certain project the startup project to test it, I need to switch the configuration properties to create a .exe file otherwise it wont run.
Is there a way to change the configuration of your projects based on what project is the startup project? So that I can instantly run this instead of manually changing the properties everytime I take a differnt startup project.
Your plan won't work the way you want. But I suppose you could add an extra project that can load the correct lib file and launch it. Then in the debug settings of each lib project specify that debugging will launch your extra project with the path to the lib to load as an argument.
It will require a small custom project, but will make your life easier. It'll look something like this in the debug settings:
I'm creating a static lib on Mac OS X for one of our customers, as well as a small cmd line app to test the static lib. The cmd line project has 2 extra library search paths, which meant I was linking to the Debug version in Release mode and just about went crazy, so I tried to get rid of these two paths, but I couldn't find where they were specified. I was looking in the project info, but it turns out they were specified in the target info.
I don't understand the distinction?! Why there are 2 sets of settings, which are essentially the same?! Can someone please enlighten me?
A project can contain multiple targets. For example, an app I write has four - the app itself, a Quick Look plugin, a framework and a bundle that contains Mac OS 10.6-specific functionality that can be dynamically loaded in.
Project settings apply to every single target in the project. Each target can then override individual settings if they need to - for instance, my project's Target SDK is set to 10.5, but the 10.6-specific bundle has it's Target SDK set to 10.6.
In some instances, some settings don't make sense to be in Project Settings - one of these, I guess, is search paths.
You often have multiple targets in a single project - for instance, you might have a framework project with a target for building as a dynamic .framework bundle, and a target for building a static lib. Or your app might have a target for building the app itself, and a target for building some helper command-line tool that it needs to install.
Wherever possible, I'd suggest changing settings at the highest level (in the project settings, and simultaneously changing debug & release configurations), and only customizing the target settings when necessary. Even better, move as many settings as possible into xcconfig files, which seem a much more explicit way of specifying your build setup.
Preface: you ship targets. Your end products are targets. Not projects. Think of a project as the umbrella above multiple targets.
For a more realistic example assume both Uber and Lyft were being developed by the your (umbrella) company.
The company has the following three environments:
Debug
QA
Release
The Debug and Release configs come out of the box with every new project you create. You can create as many additional configs as you want
This would require 3 configurations. To add a QA configuration follow the tutorial here
Did I apply this to the target or project?
I applied it to the project.
Ok so configs are only for projects and not for targets. Right?
Incorrect! It's confusing I know. You have to think of the project as a big container where you create your configs in there.
Then for each target (not project), for following tabs:
General, Resource Tags, Build rules, Info:
There is no difference between different configs
Signing and Capabilities tab:
You can switch between teams and sign it with a different team. This is useful if you want to sign your beta builds with your enterprise certificate but sign your Appstore build with app store certificate.
Build Settings tab:
For almost every variable in this section you can give a different value based on the config. Common Build Settings to customize are:
Architectures - 'Build Active Architecture only'
Build Options 'Debug Information Format'
Packaging
Set the plist you want per configuration.
Change the bundle identifier (Packaging >> Product Bundle Identifier). If you switch values for a field then in the plist you'll see as:
Signing
Code Signing identity
Code Signing Style (Manual or Automatic)
Development team
Provisioning Profile
Apple Clang - Optimization Level
If the values are different then the row's value would be
<Multiple values>
and you basically have to expand that value to see what value is given for debug and what value is given for Release or QA config.
If all the values are the same then you'll just see the value that is given to all of them. By default the values are the same.
Build Phases:
There's no out GUI way of switching based on configuration. However you can still do run certain commands based on the configuration. Example:
if [ "${CONFIGURATION}" = "Debug" ]; then
"${PODS_ROOT}/SwiftLint/swiftlint" autocorrect
"${PODS_ROOT}/SwiftLint/swiftlint"
fi
Summary
Long story short, this allows you to have 2 different apps (targets) with the same code (project), in 3 different environments (dev, QA, release). You create the different environments using configurations.
To learn more on this I highly recommend you to read more about this in depth and understand what configuration files (xcconfig) is. It's much more simpler than you think. It's mainly a key value pair:
AppCoda - Using Xcode Configuration (.xcconfig) to Manage Different Build Settings
NSHipster - Xcode Build Configuration Files
I have an Xcode project (A) referencing another project (B). By default (as far as I understand it) Xcode will implicitly build the configuration for the B dependency that matches the configuration of the A's target (e.g., "Debug"). But what if I want, say, A to build as "Debug" and the B to build as "Release"? How would I go about specifying that in Xcode?
I don't know of any easy approach, but you can brute-force it by calling xcodebuild directly for the dependency with a "Run Script" build phase.
I know it was just an example, but if your real goal is that the sub-project be a Release (no symbols) build, then you may have a better experience by just building the sub-project into a library or framework and checking the resulting binary into your version control system. Whenever I have a piece of the system that seldom changes and that I don't want debug symbols for, I go ahead and build it as a static library and check it in. I often go ahead and move the code elsewhere as well (with a README file with the .a that says where the code is and how it was built). This saves time on both build and checkout and is invaluable for large projects in my experience.
This might help: if the configuration of the project A is not found, Xcode will build Release config as a fallback (or maybe the first config of the list).
Then you can "force" the link using this tip: Xcode custom build configuration causes "library/file not found" for static libraries
Yes, this is not naturally supported by Xcode; when you build a target, it builds one configuration of itself and of all dependent targets.
The workaround, as Rob mentioned, is to have a dependent target that's an Aggregate Target type that comprises a single Run Script build phase, which simply invokes xcodebuild -configuration Release (or whatever).
You can specify the default 'fallback' configuration in the project info.
Change from:
Use 'Release' for command-line builds.
to:
Use 'Debug' for command-line builds.
And default will be 'Debug'.
Diffs of project file: