I found a command-line tool called "xctest" that apparently can run the unit tests in your project. This executable lives here:
/Applications/Xcode.app/Contents/Developer/usr/bin/xctest
When I try to run this executable on my xctest bundle, I'm using:
$ ./xctest /Users/myusername/Library/Developer/Xcode/DerivedData/MyApp-abcdefghijklmnop/Build/Products/Debug/MyAppTests.xctest
However, I get the following output:
Test Suite '(null)' started at 2013-11-14 21:16:45 +0000
Test Suite '(null)' finished at 2013-11-14 21:16:45 +0000.
Executed 0 tests, with 0 failures (0 unexpected) in 0.000 (0.001) seconds
There's no man page for xctest, as far as I can tell, but entering just ./xctest at the command-line yields:
Usage: xctest [--test Self | All | None | <TestCaseClassName/testMethodName>] <path of unit to be tested>
In particular, I'd like to be able to test just a particular method in a test class, which is why I'd like to use this xctest command.
I do see that there is a way to run all the tests from the command line like:
$ xcodebuild test -scheme MyApp
This runs all the unit tests and works properly (I see my unit test results, unlike when using xctest). But I'm interested in being able to run a single test method from the command-line, such as:
$ ./xctest --test MyAppTests/testExample /Users/myusername/Library/Developer/Xcode/DerivedData/MyApp-abcdefghijklmnop/Build/Products/Debug/MyAppTests.xctest
Despite what the usage message says -XCTest is the argument you need:
xctest -XCTest MyAppTests/testExample testbundle.xctest
For a direct invocation of xctest to work you may also need to set DYLD_FRAMEWORK_PATH and DYLD_LIBRARY_PATH to your built products directory. In general you need to use the same arguments and environment as Xcode does, you can see this by putting a breakpoint in one of your tests, running them through Xcode, then printing out the values of arguments and environment for [NSProcessInfo processInfo].
To avoid messing with all that note you can also modify the scheme in Xcode to run only specific tests. Under Product > Scheme > Edit Scheme select the Test action and expand the test bundle. You can use the check boxes to select the tests to run and xcodebuild's test action will then run only these tests.
Related
I'm running the following command to run my .net Core tests:
dotnet test
This runs fine. I want to now generate code coverage stats, so after following this article, I ran this:
dotnet test AI.Core.Tests.csproj
/p:CollectCoverage=true
/p:CoverletOutputFormat=cobertura
/p:CoverletOutput=TestResults\Coverage
I get the following output from this command:
C:\Users\sp4_rm\.nuget\packages\coverlet.msbuild\2.2.1\build\netstandard2.0\coverlet.msbuild.targets(23,5): error :
Index was out of range. Must be non-negative and less than the size of the collection.
[C:\Users\sp4_rm\Desktop\EVO\AI.Core\src\Tests\AI.Core.Tests.csproj]
C:\Users\sp4_rm\.nuget\packages\coverlet.msbuild\2.2.1\build\netstandard2.0\coverlet.msbuild.targets(23,5): error :
Parameter name: index
[C:\Users\sp4_rm\Desktop\EVO\AI.Core\src\Tests\AI.Core.Tests.csproj]
See screen shot below:
Has anyone got this command running? What am I doing wrong?
Ok so this was due to a school boy error in not actually have any code to test (or test case to test it) in my sample project!! Adding a couple of classes into the main projects and a couple of tests in the test project does away with this problem! (just in case anyway does the same thing as me!)
I would like to run a Makefile in IntelliJ whenever a source file changes. Therefore I installed the File Watcher plugin and added a new File Watcher for Go files. In the Watcher Settings I added make as the Program and test for the Arguments.
When I now change a .go file, I get the following error message:
make test
make: *** No rule to make target `test'. Stop.
Process finished with exit code 2
so I assume that the file watcher works in general but is somehow runs the make command in the wrong directory.
When I run make test in a terminal from the root of my project, everything works as expected and I get:
Formatting all packages...
Code analysis with go vet...
Execute test with ginkgo...
[1490735873] Cmd Suite - 1/1 specs • SUCCESS! 120.183µs PASS
[1490735873] Test Helpers Suite - 4/4 specs •••• SUCCESS! 125.046µs PASS
[1490735873] Models Suite - 5/5 specs ••••• SUCCESS! 453.456µs PASS
[1490735873] Services Suite - 16/16 specs •••••••••••••••• SUCCESS! 3.035275ms PASS
Ginkgo ran 4 suites in 855.886243ms
Test Suite Passed
What am I doing wrong - or am I missing the point of using file watchers in IntelliJ?
Solution was that I had to specify the Working Directory in Other Options. When I set this to $ContentRoot$, everything works as expected.
I have created an release app with rebar3 (beta-4).
Added some eunit tests and wrote some code.
For now I have to debug one test case to see what I have to add to make the implementation to work properly.
I found some articles about using dbg from Erlang console and I found how to write debug info from Eunit. But I need to get info from code that I have to test (the actual implementation(logic)).
Is there a way to debug Erlang code (actual source code, not the test one) when rebar3 is used with eunit argument?
I'm using tracing in terminal like there: https://aloiroberto.wordpress.com/2009/02/23/tracing-erlang-functions/
One way to do this is use rebar3 to run a shell under the test profile, then start the debugger and set up your breakpoints and such:
$ rebar3 as test shell
...
1> debugger:start().
{ok, <0.67.0>}
This will pop up the debugger GUI. Once the debugger is set up and ready, run your test under eunit:
2> eunit:test(your_test_module,[verbose]).
======================== EUnit ========================
your_test_module: xyz_test_ (module 'your_test_module')...
Assuming you set up a suitable breakpoint in the debugger, this will hit it, but you'll likely run into a problem with this approach: by default, eunit tests time out after 5 seconds, which doesn't give you much time for debugging. You need to specify a longer timeout for your test, which is why the example above shows that what's running is a test fixture named xyz_test_, which wraps the actual test with a long timeout. Such a fixture is pretty simple:
-include_lib("eunit/include/eunit.hrl").
xyz_test_() ->
{timeout,3600,
[fun() -> ?assertMatch(expected_value, my_module:my_fun()), ok end]}.
Here, the actual test is the anonymous function, which matches the return value from my_module:my_fun/0, which for this example represents the business logic under test. This example fixture sets the test timeout to one hour; you can of course set it as needed for your application.
I would like to post the results of tests for my iPhone app to my TestLink using XML-RPC.
I use Kiwi in my project, and now I want to get the results of tests. Can I know whether the conditions on my testcase passed of failed?
The results of Kiwi tests are recorded in basically the same way as the results of typical OCUnit tests, thus there shouldn't be anything special about exporting the results of Kiwi tests compared to other Xcode test frameworks. With Xcode 4, a log file is generated at:
~/Library/Developer/Xcode/DerivedData/(product_identifier)/TestResults/(timestamp).xctestresults/results.plist
This plist file has a simple format, with an array of dicts for each Kiwi spec. The "Test Identifier" and "Test Name" values are generated by Kiwi by concatenating strings in the (possibly nested) context and it declarations for each spec, and the "Test Result" value will either be Succeeded or Failed.
You might want to refer to some other questions (3 different links) on Stack Overflow that discuss exporting Xcode test results or automating/scripting testing.
If you don't mind running the Kiwi tests from the command line then you could use xctool. This allows you to output the contents of the tests that were run (and their pass/fail statuses) to a JSON file which is friendly(er) for importing into a third party ticketing system like Test Link compared to the raw XC.
After installing xctool run your tests like this:
xctool test \
-project ProjectName.xcodeproj/ \
-scheme SchemeName \
-reporter phabricator > ~/Desktop/test_results.json
That will output a JSON file which can be imported directly into Phabricator, but you could run another script after to put the JSON into in a format Test Link can integrate.
I have many test case in Microsoft test manager that works right. I must run them single.I want run a test suite that contains many Test case. but When I select a test suite the run Option is disable.
How I can run a test suite or a test plan.
Sincerely you M.bagheri
If you create a test suite with test cases inside, then you can go under the tab "Test" in MTM right click on the test suite and you can "Run" or "Run with Options". If that is not available something is wrong.
Its probably because you have not selected any test in right side panel of MTM. Select all/intended test cases from the tests available in your test suite and then click on run option, you will see the option enabled:
Since you mentioned that you want to run test in your test suite, i assume that you have targeted tests available in your test suite:
Alternate Approch:
You can use tcm.exe command line utility to run test cases: (Open visual studio command prompt and run following sample command from your
tcm run /create /title:"" /planid: /suiteid: /configid: /settingsname:"" /testenvironment:"" /collection:http://:8080/tfs /teamproject: /include
You can go through following link for more info on using tcm utility:
http://msdn.microsoft.com/en-us/library/dd465192.aspx
Following commands might come in handy for finding values for options in above mentioned tcm command:
*tcm plans /list /collection:http://[your_tfs_server_name]:8080/tfs /teamproject:[your_tfsproject_name]*
*tcm configs /list /collection:http://[your_tfs_server_name]:8080/tfs /teamproject:[your_tfsproject_name]*
*tcm suites /list /planid:[your_testplan_id_can_be_picked_from_first_command] /collection:http://[your_tfs_server_name]:8080/tfs /teamproject:[your_tfsproject_name]*
tcm help