I read the question on 'The best way to unit test Objective-C and followed the instructions, but no matter what I do, the Unit tests do not run. Actually the entire program does not run, I get the following message.
dyld: Library not loaded: #rpath/SenTestingKit.framework/Versions/A/SenTestingKit Referenced from /Users/garethlewis/work/objc/UnitTesting/build/Debug/UnitTesting
Reason: image not found
I have set the DYLD_FALLBACK_FRAMEWORK_PATH variable, and also the XCInjectBundle as well as the DYLD_INSERT_LIBRARIES and added the variable -SenTest All.
I can't have the only installation of Xcode 3.1 that Unit testing fails on.
Can someone who has managed to get Unit Testing on Xcode 3.1 working give some details on what needs to be done. It would help so much, with what I am trying to do.
You don't need to do this stuff to just run your tests.
If you're writing tests for an application, you should just need to set the Test Host and Bundle Loader build settings for your unit test bundle target and they will be run as part of your build. If you're writing tests for a framework you don't even need to do that, just make sure your test bundle links against your framework.
I assume you're actually talking about debugging your tests, not just running them. If so, it's important to give us the following information:
what kind of tests — application or framework — you're trying to debug
what environment variables you set, and what values you set them to
what arguments you set (-SenTest All should be an argument, not environment variable)
what the full error shown in your debug console is, not just the specific failure
That will help diagnose what's going on.
At first glance, it looks like you might have a typo in your DYLD_FALLBACK_FRAMEWORK_PATH because that determines where dyld will look for the SenTestingKit.framework binary if #rpath cannot be resolved. Knowing what it's set to will probably help.
(PS - It's Xcode.)
Related
I have been using Clojure, ClojureScript, lein, shadow-cljs, re-frame, reagent, Emacs, and CIDER to work on a Clojure/ClojureScript dynamic web app project.
Usually, I build the project by executing shadow-cljs watch app. It works fine. I can use the application and watch changes.
Currently, I am working on a Continuous Integration project via GitHub Actions. In this new environment, I wanna use a different command: shadow-cljs compile app.
But, I am having problems. Both on GitHub Actions env and in my local env (which is reproducing the steps on GitHub Actions).
The result returned by the command is quite weird. First, it says the compilation went well, Build completed, and the terminal displays some non-harmful warnings:
shadow-cljs compile app
shadow-cljs - config: /Users/pedro/projects/my_project/shadow-cljs.edn
WARNING: random-uuid already refers to: #'clojure.core/random-uuid in namespace: portal.runtime.browser, being replaced by: #'portal.runtime.browser/random-uuid
[:app] Compiling ...
SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
WARNING: abs already refers to: #'clojure.core/abs in namespace: day8.re-frame-10x.inlined-deps.garden.v1v3v10.garden.color, being replaced by: #'day8.re-frame-10x.inlined-deps.garden.v1v3v10.garden.color/abs
WARNING: abs already refers to: #'clojure.core/abs in namespace: garden.color, being replaced by: #'garden.color/abs
[:app] Build completed. (1121 files, 27 compiled, 8 warnings, 45.85s)
------ WARNING #1 - :redef -----------------------------------------------------
(... omitted ...)
Unfortunately, the terminal stays like this forever. It behaves like the watch command... But, compile is not supposed to be like this.
The terminal does not finish the process. There is no exit. Even though the build is complete, as stated in the last line before WARNING.
Feels like a deadlock situation. The terminal is still responsive because I can cancel the process with Control-c. But, besides canceling it, it is frozen. This can be catastrophic on GitHub actions since you are going to be paying for an ongoing process.
On shadow-clj.edn, I tried commenting out compilation-options:
;:compiler-options {:optimizations :none}
But it did not make any difference. The same problem happens with shadow-cljs release app.
After finding a GitHub issue with a similar problem, I also tried:
:js-options {:resolve {"highlight.js" {:target :npm :require "highlight.js/lib/core"}}}
One of my hypotheses is that highlight.js is causing some trouble while parsing strings and/or files...
I am also afraid the problem could be some REPL being fired up by some dependency and blocking the process for the exit... But I am not sure where to look for it.
Why is this happening? What could be causing this? How can I solve it?
;;;;
UPDATE:
After #thomasheller kind answer, I tried some things. I believe all of the hints can be discarded for the present situation, except for the macros:
1 - The project does not use user.clj
2 - Build hooks is the default setting which is basically a comment:
:build-hooks [;; this will create a build report for every release build
;; which includes a detailed breakdown of the included sources
;; and how much they each contributed to the overall size:
#_(shadow.cljs.build-report/hook
{:output-to "build-reports/report.html"})]
3- There is no shadow-cljs start, server, or watch process going before trying to compile. The problem of an endless compile app happens just after re-starting my MacBook, on the CI, in others' people's PC... So, it is unrelated.
Ok. Now, let's talk about the macros... The project's main repository has 6 defmacros - according to a git grep (I have not checked the dependencies).
I am suspicious about one of the macros which is involved in reading files:
(defmacro slurp [file & [default]]
(if (.exists (io/file file))
(clojure.core/slurp file)
default))
Why does this exist? Well, it is being used inside another macro that reads config files related to environment variables:
(defmacro read-open-config [env-var]
(clojure.edn/read-string (slurp (str "config/" (System/getenv env-var) "-open.edn"))))
We have multiple config files for different purposes (I know it is not the standard practice...).
It feels like too much dynamicity...
First of all a few clarifications.
compile produces a development build, no optimizations are applied. Setting anything related to that does nothing.
CI systems should likely use release. This avoid including all the development related code (eg. highlight.js, re-frame-10x).
A dependency cannot "fire up a REPL". The shadow-cljs "server mode" provides it, which compile or release don't normally enter.
highlight.js errors would fail earlier with a visible error, you would not get to "Build completed."
This could be happening for a variety of reasons, very hard to debug without seeing the build config/setup. A few guesses:
You have a user.clj on the classpath, which starts additional "stuff" when the process is launched and this namespace is loaded? Clojure will load this unconditionally and shadow-cljs cannot prevent whatever that may do.
You have configured :build-hooks that launches additional stuff. It could launch an addition css watch process that doesn't exit?
You use a macro that launches additional stuff without shutting it down?
You previously started a shadow-cljs start, server or watch process? Coupled with the above that may then wait for the server to stop. On your local machine is the watch maybe still running? You can prevent the attempted server connect via npx shadow-cljs release build --force-spawn, but that is only useful if it is actually running.
Normally shadow-cljs will shut down after compile or release. I'm not aware of anyone ever having any issues with this after a successful build. The issue you linked is not related to yours, since that never got to a completed build.
After a long research process, it was discovered that the problem came from tests inside a private dependency. Apparently, some tests with asynchronous processes profile were holding the compilation process from finishing.
After commenting-out some olds tests, the compilation happened as expected.
I'm evaluating ginkgo at the moment - I very much like the BDD style.
However I'm unable at the moment to get the VS Code debugger to work with the framework. The official VS-Code extension provides test-by-test debugging for native go tests using CodeLens. With other languages and frameworks (eg Typescript/Mocha), I've been able to debug individual test files by setting up launch.json appropriately, but have been unable to find suitable examples for go.
Does anybody have any examples of any launch.json setups for debugging ginkgo tests (or go code invoked from any other framework)?
Thanks!
After a bit of playing around I found a way forward which perhaps should have been obvious. In case it isn't I'll leave the question and this answer here:
For a package foo, a foo_suite_test.go file is generated by the gingko bootstrap command. This contains a top-level test called TestFoo which runs the rest of the tests within the package.
This does have a CodeLens run test | debug test section above it which you can use to debug the entire suite.
It's not quite as convenient as the individual CodeLens entries which appear over each native go test, but it's easy enough to isolate specific tests to run using the Gingko F prefix.
Oh, Xcode!
I'm stymied. If I create a Mac OS Cocoa app, I get a unit test target for free. But I want to write a command line tool (or even a library that I can link into a command line tool) and write some unit tests against it. I'll be darned if I can figure it out.
I've gotten far enough that I have a command line target, and a test target. I can "#testable import" my commnd line module and use the code in the test code with no errors in Xcode. However, as soon as I try to run my test (Cmd-U), I get a link error. None of the classes in the main module can be linked. What?
I've been messing with this for hours, and the Great Google has been no help. (I'm using Xcode 7, and Xcode 6 seems very different.) Can anyone help me?
Thanks.
So far the only solution I've found for this problem is to manually add all the files containing code that you want to test to unit test target manually:
This is something you wouldn't do when testing an application target. I think the fact that the command line target cannot be selected as the test host for a unit test target might be related with this issue:
Another option you have, which might require a bit more work, is to define all your logic into a Framework, and write the command line app as a consumer of the APIs it provides.
This way you can easily unit test the framework the usual way, and then write integration tests for the command line app in the form of scripts that call it and assert the results.
After struggling with this here is the solution that worked for me:
Step 1: Add a testing bundle. Editor > Add Target, Cocoa Testing Bundle
Step 2: Edit the scheme. Product > Scheme > Edit Scheme. Select Test, click +. Under “Choose targets to test as part of this scheme”, select your test target.
Step 3: Try with you simple test
There are cases when adding the same file to both utility and the test target is not possible (some kind of loops resulting in redefinitions).
However, it's possible to create an additional application target, add all files of the utility except for main.swift to the app, and then use this app as a hosting app for the tests (and also enable "Allow testing Host Application APIs checkbox).
It seems that Logic tests require that I add application files (.m and .h) to the compile target of the test along with the test .m files. I think that this is because Application tests actually load the app side by side with the tests but I am not sure.
Is the bundle loader 'the difference' between Application and Logic tests? I am not talking about why to use them but the distinction between how they work and what they do.
Here you can find a nice description of the tests, with what they do and how they differ. Also, in the next section here, you can read about how to set them up, which reveals the technical differences between them. As you suggested, loading the app bundle seems to make the difference.
Edit: Seems that the links above no longer work, I found a legacy document here: https://developer.apple.com/legacy/library/documentation/DeveloperTools/Conceptual/UnitTesting/UnitTesting.pdf
Following Chris Hanson's blogs and Apple's Automated Unit Testing with Xcode 3 and Objective-C I have started implementing unit tests for my projects. However, I use a lot of plug-ins (loadable bundles) and I can't quite figure out how to test them.
I figured I would use the approach Chris Hanson described for testing frameworks.
I started with a Cocoa Bundle project, added a principal class and changed the type to plugin.
Then I added the unit test bundle, add the plugin as a direct dependency (Apple's instructions) and set the following build settings:
Bundle Loader: $(BUILT_PRODUCTS_DIR)/CocoaPlugin.plugin/Contents/MacOS/CocoaPlugin
Test Host: $(BUNDLE_LOADER)
The problem is that as soon as I've done that and build the test target, I get this message:
error: Test host '/Users/elisevanlooij/Documents/Plug-ins/CocoaPlugin/build/DebugCocoaPlugin.plugin/Contents/MacOS/CocoaPlugin' exited abnormally with code 127 (it may have crashed). [code 126 in another plugin]
I had hoped that adding the otest custom executable would help, but unfortunately not. I really hope someone can help because not being able to unit test my plugin really puts a cramp in my testing lifestyle.
Take a step back. Your Bundle Loader setting is erroneous and adding a custom executable is not going to affect compilation of a unit-test bundle.
You need to get your unit-test bundle to build without errors (and warnings!), and your tests will run automatically (you do have at least one valid SenTestCase class with at least one valid test method, right?).
So,
are you saying that your test-bundle compiles without warnings and you have written some tests using classes and methods from your plugin? If so you must have some how taken care of loading the plugin-bundle into the unit-test-bundle and defining some kind of API, as the plugin-bundle doesn't have any public headers, right?
see Apple docs here
Loading plugins into plugins (essentially what you are trying to do) is not easy and they are not magically 'linked' at compile time like the frameworks in the Chris Hanson Blog that you refer too. They wouldn't be plugins if they were.
The simplest way to go is to not actually test your plugin at all but add the files you want to test directly to the unit-test bundle. At least this way you can get on with testing your code without fiddling about with dynamically loading bundles.
But if this isn't satisfactory, you can get what you are trying to do to work with a little effort - you should definitely add tests to verify that your plugin is loaded and that the symbols you think are available REALLY are available. Once your tests build ok you should follow Chris Hanson's other excellent blog on debugging unit test bundles showing you how to step thru your tests in the debugger - you should be able to track down any errors.
Your
Bundle Loader: $(BUILT_PRODUCTS_DIR)/CocoaPlugin.plugin/Contents/MacOS/CocoaPlugin
is correct. It means that when
linking your test bundle you do not include the classes under test there, and they will
be looked up from CocoaPlugin. It is a compile time setting and should cause your test
bundle to compile/link sucesfully. (See -bundle_loader in man ld)
Your
Test Host: $(BUNDLE_LOADER)
is incorrect. Your test host should be either an application (with a NSApplicationMain called from main method) or not set. This TEST_HOST setting is a runtime setting to run your unit tests. You basically have two options:
Do not set TEST_HOST, and load your plugin from your test bundle. For example you can do this using the initlaize method.
Create a dummy test_host application that will load your plugin, and then call NSApplicationMain, and use this app as your TEST_HOST.
The +initalize method for your test bundle to load the plugin would look like this:
+ (void)initialize
{
NSBundle* bundle = [NSBundle bundleWithPath:pathToPlugin];
[bundle load];
NSLog(#"Loaded:%#\n",bundle);
}
The main method in your dummy_test host app can look like this:
int main(int argc,const char** argv)
{
NSBundle* bundle = [NSBundle bundleWithPath:pathToPlugin];
[bundle load];
NSLog(#"Loaded:%#\n",bundle);
return NSApplicationMain(argc,argv);
}
Other ideas for testing plugins:
use an independent bundle: Do not specify either BUNDLE_LOADER or TEST_HOST and put your classes from the plugin also into the unittest bundle.
put your test cases into the plugin, and try to get that unittest. Just weak link SenTestingKit to your plugin and add a script phase with: TEST_RIG=/Developer/Tools/otest "${SYSTEM_DEVELOPER_DIR}/Tools/RunUnitTests".