I have a number of unit tests written for my project, executed with CTest. I would like to integrate the results into my TeamCity build. I've downloaded and set up the plugin for my testing framework (Boost Test).
The problem that I have run into is that the tests run with CTest output to Testing/Temporary/LastTest.log, whereas TeamCity is trying to read the results from standard out. To get around this, my testing step is.
make test
cat Testing/Temporary/LastTest.log
which works, but feels like a hack.
Is there any way to get TeamCity to read from this file in addition to standard out? Alternatively, is there any way to tell ctest to output to standard out in addition to this LastTest.log file?
This question is similar, but I would like it to work for all output rather than just on failure: CMake: setting an environmental variable for ctest (or otherwise getting failed test output from ctest/make test automatically)
Teamcity has additional build features which allow to process CTest reports. I am not sure if it'll work or not but you could try adding an additional build feature in your build step to read CTest report.
Related
I'm trying to learn several things at once (arguably my first problem...), namely: unit testing with Catch2 and building with CMake.
In the course of my investigations, CTest appeared on the radar as a pre-baked way of managing tests within CMake, and seems to 'support' Catch2.
While things seem to build okay, I can't run my tests as automatically as I'd hope.
Specifically, I have a source tree, which at some point contains the library I'm testing, and I'd like to be able to sit at the top of the tree and execute some sort of 'run my tests' command (and ideally run them as part of the full build, but that's for another day).
So here's my CMakeLists.txt file (L:\scratch\shared\testeroolib\CMakeLists.txt) for the library of interest:
cmake_minimum_required(VERSION 3.5)
project(testeroolib)
add_library(${PROJECT_NAME} STATIC src/testeroolib.cpp)
target_include_directories(${PROJECT_NAME} PUBLIC include)
set(PROJECT_TEST ${PROJECT_NAME}_test)
add_executable(${PROJECT_TEST} test/catch2_main.cpp)
target_link_libraries(${PROJECT_TEST} PRIVATE Catch)
target_link_libraries(${PROJECT_TEST} PRIVATE ${PROJECT_NAME})
enable_testing()
add_test(NAME TesterooLibTest COMMAND ${PROJECT_TEST})
If I do the naive thing and run ctest from the same location I run cmake, I get:
L:\scratch>ctest
*********************************
No test configuration file found!
*********************************
...
or
L:\scratch>ctest .
Test project L:/scratch
No tests were found!!!
From what I've read elsewhere, make test would do the trick with GCC and friends, but I'm using VS.
So here, the advice seems to be that I should use a build target of ALL_TESTS but that doesn't do the trick for me.
L:\scratch>cmake --build BUILD --target ALL_TESTS
...
MSBUILD : error MSB1009: Project file does not exist.
Switch: ALL_TESTS.vcxproj
Of course, I can just run the test:
L:\scratch>BUILD\shared\testeroolib\Debug\testeroolib_test.exe
===============================================================================
All tests passed (1 assertion in 1 test case)
I'm hoping I've made some tiny snafu, but there is every chance I've got completely the wrong end of the stick here!
I believe I've found the (two-part) answer.
The first part is that it's no good running ctest from the top level, you need to run it from the build folder. With hindsight, that's pretty obvious :(
cmake -S . -B BUILD
cmake --build BUILD
cd BUILD
ctest
The less obvious part I found in this answer: https://stackoverflow.com/a/13551858/11603085.
Namely, the enable_testing() call needs to be in the top-level CMakeLists.txt, not the one further down that actually builds the library.
I believe the root cause of your problem is the missing Debug/Release configuration info which cmake needs when you configured using Visual Studio. Try:
ctest -C Debug
for example.
I'm very new to App-V, which is evaluated in my office.
I have a Selenium test suite written in JUnit5 and can launch it as gradle test task using gradle-wrapper. My final goal is to run this on App-V5.1 virtualized environment, similar as this question.
As followed the link which was mentioned the answer, I tried to launch cmd.exe within the App-V environment, and it seeded works. Then, I tried to do this:
./gradlew --no-daemon clean test
Then the testClasses phase works perfectly, but in the test phase, I got an error like:
Could not write standard input into: Gradle Worker 1.
java.io.IOException: The pipe is being closed
...
(Sorry I couldn't show you the actual error log due to security reason, but it is similar to this question.)
May I wrong something? What's the right way to launch a gradle test in App-V env?
Have you tried launching cmd.exe from within the virtual bubble? I find the best way to do this is to create a shortcut to cmd.exe during sequencing and use this to troubleshoot.
If your process works within the bubble, the solution may be as simple as allowing Local Interaction. Have a read here about that.
Bamboo gives us the ability to run phpunit tests, but likely given a lower popularity, nothing for phpspec.
I suspect however (Googling) that it must be possible!
https://revive.beccati.com/bamboo/browse/PHP-PHPSPEC-813/test
Has anyone successfully fed phpspec tests into Bamboo?
Thanks.
One option is to run the phpspec test as a Script Task. If the task returns a non-0 exit code, the build fails. phpspec obligingly returns a 0 exit code only when all your tests pass.
In order to do this you would need to ensure that phpspec is available to your Bamboo build. If you're using Composer, you can add it to the require-dev section of your composer.json file.
Then, in the Plan Configuration, under Default Job (or some other job), you can add a new task of type Script. This Script Task can then make a call to phpspec:
vendor/bin/phpspec run 1>&2
You may also want (as above) to redirect the output to stderr because Bamboo seems to suppress any output on stdout. This will then allow you to see the output of phpspec in your Bamboo log.
The answer was to run the test with the junit formatter. Bamboo has built-in support for the former, and this made the tests run smoothly.
I have a TeamCity build that sometimes fails too early.
What I mean by that is that the first few steps are for "provisioning" (setting up the testing environment) and the testing of my code itself comes later.
Sometimes (for whatever reason) the build fails during one of the "provisioning" steps. This is not a problem since running the build again usually works fine.
But - the "changes" are not passed along to the next run of the build.
I am using this command as part of my build to output the "changes" that came from my codebase:
copy "%system.teamcity.build.changedFiles.file%" changelog.txt
So I need a way to tell TeamCity "hey, ignore the last run, that failure doesn't count because it didn't test my code, I want the next run to contain the same 'changes' in system.teamcity.build.changedFiles.file"
How can I do that?
Have you tried build chains with dependencies? They can be set up to only execute if the build (including tests) is successful: http://blog.jetbrains.com/teamcity/2012/04/teamcity-build-dependencies-2/
I am new to CI and would like your thoughts and input on how to go about my problem. I would like to first start off that I have been wrestling with this for 2 days(and I don't have that much background in sys ad) so please play nice?(I am mainly a front-end web dev) :)
Basically my plan was to install jenkins then make a CI env with these steps:
poll for any changes to github
if there are, run the build script:
a. migrate the development and test dbs?(does that mean i have to put the config/database.yml in my repo?)
b. run cucumber
c. if all tests pass go to 3, else fail
run any rake setup stuff
run the server(deploy)
I have done some of the stuff by cheating:
in my local, i switch my rvm to the correct one i need(rvm use 1.8.7-p174#mygemset)
run jenkins(java -jar jenkins.war) so that it gets the RVM ruby as default
run spork in a separate terminal(because for some reason my cucumbers don't run without spork - that's another problem)
build the project manually by clicking Build
so basically, I want to automate these stuff. Maybe what I need is a set of steps to follow(general or specific, depending on your taste) so I can setup my CI up and running.
Keep in mind that my "cheats" won't do as I want to test different projects with different setups and the startup cheat just won't do. Currently, my project build was successful because all I did was to run cucumber(and all my cukes pass). I want it to be able to deploy after it passes so maybe some help there also? Thanks
Okay I will try and help you as best I can.
poll for any changes to github
This can be easily done with the Github Plugin located here
if there are, run the build script: a. migrate the development and test dbs?(does that mean i have to put the config/database.yml in my
repo?) b. run cucumber c. if all tests pass go to 3, else fail
Then all you would is run the build script you have configure in the in the build from
Select "Add Build Step" -> "Execute shell".
You can either do that which is probably what I would do because when you create build you want them to be portable so you can start up in new jenkins instances, so you dont have to setup your build machine, with build specific files.
Then you run your tests, if they fail the build should fail regardless here is some information on running ruby on rails tests. if you need to manually fail a build in a script based on a result usually exiting a script with non-zero will fail the build. If not continue and run your rake and deployment scripts.
Just a few notes on Jenkins it wont do everything for you but if you can do it manaually Jenkins can automate it. So anything you have setup running manually with a little bit of effort you can get up and running automated with Jenkins
Here is another answer you might find helpful in your general setup and ideology behind Jenkins.
Goodluck!