It seems that Logic tests require that I add application files (.m and .h) to the compile target of the test along with the test .m files. I think that this is because Application tests actually load the app side by side with the tests but I am not sure.
Is the bundle loader 'the difference' between Application and Logic tests? I am not talking about why to use them but the distinction between how they work and what they do.
Here you can find a nice description of the tests, with what they do and how they differ. Also, in the next section here, you can read about how to set them up, which reveals the technical differences between them. As you suggested, loading the app bundle seems to make the difference.
Edit: Seems that the links above no longer work, I found a legacy document here: https://developer.apple.com/legacy/library/documentation/DeveloperTools/Conceptual/UnitTesting/UnitTesting.pdf
Related
[Note: This question is, I think, dealing with the consequences of what prompted this never answered question]
I have a workspace that includes both a framework target and an application target. When I run the application many messages of the following form come to the console: Class "C" is implemented in both "Binary1" and "Binary2". One of the two will be used. Which one is undefined.
Here is a sampling of those messages:
objc[65093]: Class FIRAIdentifiers is implemented in both /Users/Robert/Library/Developer/Xcode/DerivedData/GenerationOfNow-bmsridmvnbtgfiduzqqomicqvsns/Build/Products/Debug-iphonesimulator/VerticonsToolbox.framework/VerticonsToolbox (0x10804bd20) and /Users/Robert/Library/Developer/CoreSimulator/Devices/33628599-9570-4784-B324-DAC383435F75/data/Containers/Bundle/Application/26411B80-FB71-44E9-AA64-05FEF20B9F08/GenerationOfNow.app/GenerationOfNow (0x10674a150). One of the two will be used. Which one is undefined.
objc[65093]: Class FIRASearchAdReporter is implemented in both /Users/Robert/Library/Developer/Xcode/DerivedData/GenerationOfNow-bmsridmvnbtgfiduzqqomicqvsns/Build/Products/Debug-iphonesimulator/VerticonsToolbox.framework/VerticonsToolbox (0x10804bd70) and /Users/Robert/Library/Developer/CoreSimulator/Devices/33628599-9570-4784-B324-DAC383435F75/data/Containers/Bundle/Application/26411B80-FB71-44E9-AA64-05FEF20B9F08/GenerationOfNow.app/GenerationOfNow (0x10674a1a0). One of the two will be used. Which one is undefined.
objc[65093]: Class FIRAZeroingWeakContainer is implemented in both /Users/Robert/Library/Developer/Xcode/DerivedData/GenerationOfNow-bmsridmvnbtgfiduzqqomicqvsns/Build/Products/Debug-iphonesimulator/VerticonsToolbox.framework/VerticonsToolbox (0x10804bde8) and /Users/Robert/Library/Developer/CoreSimulator/Devices/33628599-9570-4784-B324-DAC383435F75/data/Containers/Bundle/Application/26411B80-FB71-44E9-AA64-05FEF20B9F08/GenerationOfNow.app/GenerationOfNow (0x10674a218). One of the two will be used. Which one is undefined.
All of the duplicated class definitions come from Firebase frameworks which were installed into my workspace via the Pod file demonstrated in this question. Here is a screen shot of my workspace's Navigator:
Notice that both the framework target (VerticonsToolbox) and the application target (GenerationOfNow) are referencing the pods.
It seems to me that the proper way for a framework to be built is that it should not embed whatever frameworks it is linked against. It should be the responsibility of whatever application uses that framework to pull in the other dependencies. And indeed, when I examine the build phases for the framework VerticonsToolbox there is no option for specifying embedded binaries whereas there is for the application GenerationOfNow.
So, I am at a lose as to how to proceed. I suspect that what is happening is a result of the things that are put in place when pod install is executed. Can anyone advise me?
BTW: Can anyone point me to a good write up on how Xcode builds, what the various settings are, what tools there are for examining the binaries, etc? With Xcode everything is fine until it isn't and then there is this big, mysterious soup of stuff. Jeez!
I've created some unit tests in my Xcode 5.1.1 project. I can understand that the "t" in these icons means "test":
—but what does "rT" mean?
The standard way to do things in SenTestingKit/OCUnit/XCTest is to declare your tests in code. If you do, Xcode will discover them statically (ie. not at runtime) using the index. Once Xcode these tests are discovered, they show up in the test navigator with a "T" icon. So far so good.
Now, the SenTestingKit/OCUnit/XCTest frameworks also allow you to create tests on the fly at runtime. Some of our users make creative user of this capability, perhaps to wrap an external testing system or to create tests to represent a dynamic set of data. Xcode cannot discover these tests statically, and only find out about their existence when they are returning resutls during a test run. When discovered, they will show up in the test navigator with a "RT" icon. "RT" being short for "runtime discovered tests".
Finally. If there's anything wrong / unusual about your project that prevents indexing from completing or from properly parsing your test classes, then your tests wouldn't be statically discovered. You may still successfully build and run them, in which case Xcode would end up treating them as runtime discovered tests, and give them the "RT" icon.
Found that here:XCode 5 Testing symbol "rT" means what?
You can also check Apple's information on this:https://developer.apple.com/library/ios/recipes/xcode_help-test_navigator/Recipe.html
I understand this is a subjective question and, as such, may be closed but I think it is worth asking.
Let's say, when building an application using TDD and going through a refactor, a library appears. If you yank the code out of your main application and place it into an separate assembly, do you take the time to write tests that cover the code, even though your main application is already testing it? (It's just a refactor.)
For example, in the NerdDinner application, we see wrappers for FormsAuthentication and MembershipProvider. These objects would be very handy across multiple applications and so they could be extracted out of the NerdDinner application and placed into their own assembly and reused.
If you were writing NerdDinner today, from scratch, and you noticed that you had a grab-bag of really useful wrappers and services and you bring them into a new assembly, do you create new tests that fully cover your new assembly--possibly having repeat tests? Is it enough to say that, if your main application runs green on all its tests, your new assembly is effectively covered?
While my example with NerdDinner may be too simplistic to really worry about, I am more thinking about larger APIs or libraries. So, do you write tests to re-cover what you tested before (may be a problem because you will probably start with all your tests passing) or do you just write tests as the new assembly evolves?
In general, yes, I'd write tests for the new library; BUT it's very dependent upon the time constraints. At the least, I'd go through and refactor the unit tests that exist to properly refer to the refactored components; that alone might resolve the question.
Following Chris Hanson's blogs and Apple's Automated Unit Testing with Xcode 3 and Objective-C I have started implementing unit tests for my projects. However, I use a lot of plug-ins (loadable bundles) and I can't quite figure out how to test them.
I figured I would use the approach Chris Hanson described for testing frameworks.
I started with a Cocoa Bundle project, added a principal class and changed the type to plugin.
Then I added the unit test bundle, add the plugin as a direct dependency (Apple's instructions) and set the following build settings:
Bundle Loader: $(BUILT_PRODUCTS_DIR)/CocoaPlugin.plugin/Contents/MacOS/CocoaPlugin
Test Host: $(BUNDLE_LOADER)
The problem is that as soon as I've done that and build the test target, I get this message:
error: Test host '/Users/elisevanlooij/Documents/Plug-ins/CocoaPlugin/build/DebugCocoaPlugin.plugin/Contents/MacOS/CocoaPlugin' exited abnormally with code 127 (it may have crashed). [code 126 in another plugin]
I had hoped that adding the otest custom executable would help, but unfortunately not. I really hope someone can help because not being able to unit test my plugin really puts a cramp in my testing lifestyle.
Take a step back. Your Bundle Loader setting is erroneous and adding a custom executable is not going to affect compilation of a unit-test bundle.
You need to get your unit-test bundle to build without errors (and warnings!), and your tests will run automatically (you do have at least one valid SenTestCase class with at least one valid test method, right?).
So,
are you saying that your test-bundle compiles without warnings and you have written some tests using classes and methods from your plugin? If so you must have some how taken care of loading the plugin-bundle into the unit-test-bundle and defining some kind of API, as the plugin-bundle doesn't have any public headers, right?
see Apple docs here
Loading plugins into plugins (essentially what you are trying to do) is not easy and they are not magically 'linked' at compile time like the frameworks in the Chris Hanson Blog that you refer too. They wouldn't be plugins if they were.
The simplest way to go is to not actually test your plugin at all but add the files you want to test directly to the unit-test bundle. At least this way you can get on with testing your code without fiddling about with dynamically loading bundles.
But if this isn't satisfactory, you can get what you are trying to do to work with a little effort - you should definitely add tests to verify that your plugin is loaded and that the symbols you think are available REALLY are available. Once your tests build ok you should follow Chris Hanson's other excellent blog on debugging unit test bundles showing you how to step thru your tests in the debugger - you should be able to track down any errors.
Your
Bundle Loader: $(BUILT_PRODUCTS_DIR)/CocoaPlugin.plugin/Contents/MacOS/CocoaPlugin
is correct. It means that when
linking your test bundle you do not include the classes under test there, and they will
be looked up from CocoaPlugin. It is a compile time setting and should cause your test
bundle to compile/link sucesfully. (See -bundle_loader in man ld)
Your
Test Host: $(BUNDLE_LOADER)
is incorrect. Your test host should be either an application (with a NSApplicationMain called from main method) or not set. This TEST_HOST setting is a runtime setting to run your unit tests. You basically have two options:
Do not set TEST_HOST, and load your plugin from your test bundle. For example you can do this using the initlaize method.
Create a dummy test_host application that will load your plugin, and then call NSApplicationMain, and use this app as your TEST_HOST.
The +initalize method for your test bundle to load the plugin would look like this:
+ (void)initialize
{
NSBundle* bundle = [NSBundle bundleWithPath:pathToPlugin];
[bundle load];
NSLog(#"Loaded:%#\n",bundle);
}
The main method in your dummy_test host app can look like this:
int main(int argc,const char** argv)
{
NSBundle* bundle = [NSBundle bundleWithPath:pathToPlugin];
[bundle load];
NSLog(#"Loaded:%#\n",bundle);
return NSApplicationMain(argc,argv);
}
Other ideas for testing plugins:
use an independent bundle: Do not specify either BUNDLE_LOADER or TEST_HOST and put your classes from the plugin also into the unittest bundle.
put your test cases into the plugin, and try to get that unittest. Just weak link SenTestingKit to your plugin and add a script phase with: TEST_RIG=/Developer/Tools/otest "${SYSTEM_DEVELOPER_DIR}/Tools/RunUnitTests".
I read the question on 'The best way to unit test Objective-C and followed the instructions, but no matter what I do, the Unit tests do not run. Actually the entire program does not run, I get the following message.
dyld: Library not loaded: #rpath/SenTestingKit.framework/Versions/A/SenTestingKit Referenced from /Users/garethlewis/work/objc/UnitTesting/build/Debug/UnitTesting
Reason: image not found
I have set the DYLD_FALLBACK_FRAMEWORK_PATH variable, and also the XCInjectBundle as well as the DYLD_INSERT_LIBRARIES and added the variable -SenTest All.
I can't have the only installation of Xcode 3.1 that Unit testing fails on.
Can someone who has managed to get Unit Testing on Xcode 3.1 working give some details on what needs to be done. It would help so much, with what I am trying to do.
You don't need to do this stuff to just run your tests.
If you're writing tests for an application, you should just need to set the Test Host and Bundle Loader build settings for your unit test bundle target and they will be run as part of your build. If you're writing tests for a framework you don't even need to do that, just make sure your test bundle links against your framework.
I assume you're actually talking about debugging your tests, not just running them. If so, it's important to give us the following information:
what kind of tests — application or framework — you're trying to debug
what environment variables you set, and what values you set them to
what arguments you set (-SenTest All should be an argument, not environment variable)
what the full error shown in your debug console is, not just the specific failure
That will help diagnose what's going on.
At first glance, it looks like you might have a typo in your DYLD_FALLBACK_FRAMEWORK_PATH because that determines where dyld will look for the SenTestingKit.framework binary if #rpath cannot be resolved. Knowing what it's set to will probably help.
(PS - It's Xcode.)