I'm trying to run the xctest tests for IOS, but I have a problem when I want to run a single class, in the testlab firebase, but all the classes that I have in my project are still running, and I need to run a single class, this is possible?
It is possible if you're using the gcloud command-line tool to create your test matrix.
Specify the tests you wish to run in a copy of your .xctestrun file (same as how you would do this when running tests locally using XCode), and pass that modified xctestrun file using:
gcloud firebase test ios run --test=<your test zip> --xctestrun-file=<your customized .xctestrun file>
Im not sure if I completely understand your question, but i think what you are looking for is this:
xcodebuild test -workspace [MyApplication.xcworkspace] -scheme [yourScheme] -destination 'platform=iOS Simulator,name=iPhone X,OS=11.4' -only-testing:[Target/Class/testExample]
Where you do not have to enter testExample, only class and it will run whole class, and if you add /testMethod it will run only that test. Here you can check for more info.
Another way would be creating a scheme where you would select only tests you want to run, and then you would run only that scheme. Its pretty simple to do too. Take a look here: http://artsy.github.io/blog/2016/04/06/Testing-Schemes/
Related
I'm making macOS unit testing bundle (integration test for server software):
xcodebuild -project MyApp.xcodeproj -scheme MyApp.macOS build-for-testing
As result, xcodebuild generates xctest bundle MyAppTests.xctest and xctestrun file MyApp.xctestrun.
Now I can deploy xctest bundle on CI server and execute integration tests (by default targeted to release candidate version of server software).
xcodebuild test-without-building -xctestrun MyApp.xctestrun
Now I want to target another server instance (i.e. from development branch).
For doing this I can provide another xctestrun file MyApp-development.xctestrun with additional settings under TestingEnvironmentVariables key.
<key>TestingEnvironmentVariables</key>
<dict>
<key>com.myapp.ServerKind</key>
<string>development</string>
...
</dict>
Launch on CI server:
xcodebuild test-without-building -xctestrun MyApp-development.xctestrun
But maintaining several xctestrun files is not a good idea. I want to have default xctestrun file and override it's settings from command line.
Is it possible to pass additional settings (or override existing settings) in xctestrun file from command line?
Thank you!
Not sure if it solves your purpose, but if we wanted to override something like a User-Defined values in the Build settings of the target, i didn't find a straight forward to alter it while running test using xctestRun. So I tried directly altering the info.plist thats available inside the xctest bundle created with "build-for-testing" option before test execution and things seems to work well.
For eg. I wanted to run the test across different partitions, so i did
defaults write <plist_file_location> PARTITION B1
and this seems to do the trick
Note : Make sure <plist_file_location> is an absolute path. Providing relative path would end up creating an entry in ~/Library/Preferences/Info.plist
I would like to skip the build process and directly run tests when I do
./gradlew connectedDevDebug
I could also use adb command but it will run test on only one deivce at a time.
There is currently no gradle tasks that just runs the tests.
But once you have installed (and run) the tests, you can (re-)run them directly with the am instrument command. To start them from your development machine's command line just run:
adb shell am instrument -w <test_package_name>/<runner_class>
You can copy the actual command from AndroidStudio's output. Just run the tests from AndroidStudio, and then scroll to the top of the test log view.
There is one caveat, am instrument does not create any test-reports. All test results are written to stdout. However you could pipe stdout into a file and create a report yourself. I.e. this tool can create an xml JUnit test report from the output of am instrument.
If you have made changes to the tests you can rebuild and install them with:
./gradlew installDebugAndroidTest
For detailed information and instructions about starting tests from the command line you can refer to the official article Test from the Command Line
I have unit tests in xcode 5.1, ios7.1, when I run them via the test command within xcode I can see gcda and gcdo files generated in the appropriate place. When I invoke the test command via the command line I only get gcdo files. It must be something to do with the __gcov_flush method that needs to be called to generate the files (I put logs in the flush method I see it in the gui logs but not the terminal logs). I used help from these SO questions
Code coverage from command line
Followed basically all of this one
I have also followed the usual advice and set
Generate Test Coverage Files: YES
Instrument Program Flow: YES
At a project level.
My command:
xcodebuild -workspace "$WORKSPACE" -scheme "$SCHEME" -configuration Debug -sdk $IPHONESIMULATOR_VERSION ONLY_ACTIVE_ARCH=YES clean build test
Is there a reason why it would work within xcode but not via the command line?
I have a series of projects within a workspace, and trying to use the following type of command to build them via the command line:
"Xcodebuild -project XXX"
or
"Xcodebuild -workspace XXX -scheme YYY"
Some of my projects build fine but others give strange errors, like "'XXX' for instance message does not declare a method with selector" or "ld: library not found for -lMyLibrary"
However all these projects build fine from within the XCode UI without issues.
Based on this is seems that the command line and UI builds are using a different toolset, but that seems like a very bad idea so I'm hoping I'm wrong about this. Or possibly there are just a few different build flags being set on the command line build.
I can try to troubleshoot the issues one by one but I'm hoping I can write a script which does the exact same type of build as the UI.
Any ideas?
Without including -configuration, xcodebuild is going to use the default configuration for each project. Generally that's Release. In Xcode, the Configuration you select will be applied to every project regardless of default.
Given your errors, your most likely problem is that you've used the build pane (why I hate the build pane for large projects), and you've made the classic mistake of applying some settings for Debug rather than all configurations.
I have added a new job in my hudson server which builds the project with a makefile.
Execute shell command:
#!/bin/bash
cd $JOB_NAME
make
My makefile looks like this
SDK_31 = iphonesimulator3.1
TARGET_DEV = myProject
TARGET_TEST = unitTest
all: debug
debug:
xcodebuild -sdk ${SDK_31} -target "${TARGET_DEV}" -configuration Debug
xcodebuild -sdk ${SDK_31} -target "${TARGET_TEST}" -configuration Debug
clean:
xcodebuild -alltargets clean
rm -rf build
But when hudson build the projects, some unit tests fail but the build is tagged as successful.
What should I have to do to have an "unstable project" ?
Best regards,
You should configure Hudson to record unit test results, by enabling the 'Publish Junit test result report' post-build action.
post-build actions http://img141.imageshack.us/img141/5136/hudsonjunit.png
Update: If you can't get JUnit XML output, you should be able to use the Text-finder plugin to change the build status:
This plugin lets you search keywords in the files you specified and use that to mark the build as success or a failure.
I found this handy ruby script by Christian Hedin that converts the output of OCUnit tests (the format used by Xcode) into JUnit xml files (the format used by Hudson).
You can grab the script on github:
http://github.com/ciryon/OCUnit2JUnit
and for an explanation of how to use it, here's his blog post about it:
http://blog.jayway.com/2010/01/31/continuos-integration-for-xcode-projects/
Basically, you pipe xcodebuild into ocunit2junit.rb with a command like this:
/usr/bin/xcodebuild -target UnitTests | /usr/local/bin/ocunit2junit.rb
and it places the xml files into a test-reports folder that it creates at the root of your project folder. Then tell Hudson to copy the test-reports/*.xml artifacts as the JUnit results and you're set.
This setup will allow Hudson to correctly identify if a unit test has passed or failed and correctly mark the stability of the build.
I've been using it for a month now and it works great. The setup was very simple.