I'm using xcodebuild to build a unit test target after an every change which is commited to the SCM and if unit tests are not passed I'd like to notify the person who's introduced the changes. The problem is that even if a unit test target's build fails xcodebuild returns 0 as a return value.
Is there any way to identify that xccodebuild's build fails?
The solution can be easy. If the unit test fail, you can write something in a file, so:
First you clean the file
Than run the unit test.
You can create a trick function or use a smart #define and if something fail , you write in the file
Than is just read the file. If have a problem, you will know
Finally, if have error, you can fire the employer!
Related
I'm trying to run the xctest tests for IOS, but I have a problem when I want to run a single class, in the testlab firebase, but all the classes that I have in my project are still running, and I need to run a single class, this is possible?
It is possible if you're using the gcloud command-line tool to create your test matrix.
Specify the tests you wish to run in a copy of your .xctestrun file (same as how you would do this when running tests locally using XCode), and pass that modified xctestrun file using:
gcloud firebase test ios run --test=<your test zip> --xctestrun-file=<your customized .xctestrun file>
Im not sure if I completely understand your question, but i think what you are looking for is this:
xcodebuild test -workspace [MyApplication.xcworkspace] -scheme [yourScheme] -destination 'platform=iOS Simulator,name=iPhone X,OS=11.4' -only-testing:[Target/Class/testExample]
Where you do not have to enter testExample, only class and it will run whole class, and if you add /testMethod it will run only that test. Here you can check for more info.
Another way would be creating a scheme where you would select only tests you want to run, and then you would run only that scheme. Its pretty simple to do too. Take a look here: http://artsy.github.io/blog/2016/04/06/Testing-Schemes/
Bamboo gives us the ability to run phpunit tests, but likely given a lower popularity, nothing for phpspec.
I suspect however (Googling) that it must be possible!
https://revive.beccati.com/bamboo/browse/PHP-PHPSPEC-813/test
Has anyone successfully fed phpspec tests into Bamboo?
Thanks.
One option is to run the phpspec test as a Script Task. If the task returns a non-0 exit code, the build fails. phpspec obligingly returns a 0 exit code only when all your tests pass.
In order to do this you would need to ensure that phpspec is available to your Bamboo build. If you're using Composer, you can add it to the require-dev section of your composer.json file.
Then, in the Plan Configuration, under Default Job (or some other job), you can add a new task of type Script. This Script Task can then make a call to phpspec:
vendor/bin/phpspec run 1>&2
You may also want (as above) to redirect the output to stderr because Bamboo seems to suppress any output on stdout. This will then allow you to see the output of phpspec in your Bamboo log.
The answer was to run the test with the junit formatter. Bamboo has built-in support for the former, and this made the tests run smoothly.
I have a TeamCity build that sometimes fails too early.
What I mean by that is that the first few steps are for "provisioning" (setting up the testing environment) and the testing of my code itself comes later.
Sometimes (for whatever reason) the build fails during one of the "provisioning" steps. This is not a problem since running the build again usually works fine.
But - the "changes" are not passed along to the next run of the build.
I am using this command as part of my build to output the "changes" that came from my codebase:
copy "%system.teamcity.build.changedFiles.file%" changelog.txt
So I need a way to tell TeamCity "hey, ignore the last run, that failure doesn't count because it didn't test my code, I want the next run to contain the same 'changes' in system.teamcity.build.changedFiles.file"
How can I do that?
Have you tried build chains with dependencies? They can be set up to only execute if the build (including tests) is successful: http://blog.jetbrains.com/teamcity/2012/04/teamcity-build-dependencies-2/
In before(:all) I set up a temporary directory (#work_dir) where the tests can place their files. I would like to remove this directory inside after(:all), but only if all the tests have been successful. If one of the tests failed I would like the directory to be left intact so I can look at the generated files.
What code can I use inside after(:all) to check whether all the tests have been successful run?
You can write custom bash script, that starts running you test suite.
After that it checks return code, if it is non-zero, it keeps files. In other case it removes.
As a second solution, you can clean your directory before running tests.
In such case, when your test suite is failed, you can go and check files.
I have a number of unit tests written for my project, executed with CTest. I would like to integrate the results into my TeamCity build. I've downloaded and set up the plugin for my testing framework (Boost Test).
The problem that I have run into is that the tests run with CTest output to Testing/Temporary/LastTest.log, whereas TeamCity is trying to read the results from standard out. To get around this, my testing step is.
make test
cat Testing/Temporary/LastTest.log
which works, but feels like a hack.
Is there any way to get TeamCity to read from this file in addition to standard out? Alternatively, is there any way to tell ctest to output to standard out in addition to this LastTest.log file?
This question is similar, but I would like it to work for all output rather than just on failure: CMake: setting an environmental variable for ctest (or otherwise getting failed test output from ctest/make test automatically)
Teamcity has additional build features which allow to process CTest reports. I am not sure if it'll work or not but you could try adding an additional build feature in your build step to read CTest report.