I'm trying to run tests on my software as it's about to be deployed via git to blueMix. Currently the pipeline is 'build stage' -> 'deploy stage', and now I'm trying to implement the 'test stage' in the middle of that process.
My test stage has a test job, and the Tester Type is simple. The command I have currently is:
#!/bin/bash
# invoke tests here
go test
, however the test stage fails and returns:
_build.sh: line 3: go: command not found
Build step 'Execute shell' marked build as failure
Evidently go isn't installed at the testing stage. Do I get the test server to install Go on each new git push to test it, or am I sorely mistaken?
Thanks!
That's correct. There are only certain tools pre-installed in the enviroment that runs the jobs. You can find them here: https://hub.jazz.net/docs/deploy_var/#resources - Everything else would have to be installed each time the job is run.
Related
Is there a way to change the build config after it gets assigned to an agent from the queue?
The workflow I'm hoping for:
change in vcs root -> queue up 'run tests' -> if Agent A, run 'run tests', if Agent B, run 'run tests in Docker'
More context: I have an instance of teamcity which I maintain, but do not have admin access to. This means I can't install what I need to on the server. We have certain installs on certain build agent servers. I created different versions of our 'run tests' build config, such that it will run on all the available agents. This is a fairly simple difference: the test suite runs regularly with the one build config (on default agent) & runs in a docker container on the others (on the other two build servers). Not all servers have docker installed.
To be clear, these build configs are not sequential to each other - they are the same build config, but different versions. I don't want both jobs listening for vcs changes - as this will kick off both build configs. I simply would like to put a condition in front of the run but after the queue, however the data flow seems to be one way (queue up job, then assign to agent).
You should be able to achieve what you need using the conditional build steps, but I would not recommend that too much because the solution would be a bit hacky:
Add both build steps, 'run tests' and 'run tests in Docker', to your build configuration.
Add a condition to 'run tests' step: Execute step if teamcity.agent.name equals <dockerless-agent-name>.
Add another condition to 'run tests in Docker' step: Execute step if teamcity.agent.name does not equal <dockerless-agent-name>.
Here's the hacky part. Add the implicit agent requirements imposed by the docker step into the buildAgent.properties file of your dockerless agent. Example: If you use docker wrapper in your 'run tests in Docker' step, it will impose the docker.server.version exists agent requirement on the build configuration. In that case, add the line docker.server.version=whaterver to dockerless agent's buildAgent.properties.
TeamCity will think that the agent that doesn't have docker can run the 'run tests in Docker' step, and will be able to assign the build to that agent. But if that happens, the 'run tests' step will be executed instead of 'run tests in Docker'. That's about it.
Additionally, if the feature of ignoring implicit agent requirements if a condition is used in a build step gets implemented, you would be able to achieve the goal without hacking it through buildAgent.properties.
We have a TeamCity build configuration which does a deploy and then runs integration tests.
Deploy system
Run test suite A
Run test suite B
Run test suite C
If test suite A fails, B and C should still be run (likewise C should run if B fails). To satisfy this, the build steps are set to run "Even if some of the previous steps failed". However, I don't want any of the tests to run if the first step to deploy the system fails.
Is there a way of terminating the build if the deployment fails, but to keep running all tests of there are individual tests which fail?
You could chain the builds together so have a build for 'Deploy the system' and then have a separate build for 'Run the tests' which has your 3 steps A,B and C in it. The second build takes a snapshot dependency on the first build which means that it will kick off when the 'Deploy' build has completed, but it won't kick off if the build fails.
The steps in the second build could then be set to run even if the previous steps fail as you have it now and they would all run.
How do I run MSTest as part of my build process in TeamCity? What are the pitfalls?
This answer is specifically for TeamCity 7.1 on Windows, but may apply to other environments.
In your TeamCity build configuration, on the General Settings page
Artifact paths: Artifacts\MSTest => MSTest
Create a new Command Linebuild step
Custom script: if not exist Artifacts\MSTest mkdir Artifacts\MSTest
Create a new MSTestbuild step
List assembly files: **\bin\**\*.Tests.dll
Results file: Artifacts\MSTest\testResults.trx
Pitfalls
Using wildcards when specifying which test assemblies to run
You can use wildcards when specifying which test assemblies to run in the MSTest build step, although it is unclear exactly how they work. A bug report has been filed.
The build process doesn't stop when tests fail
Be aware that if some of your tests fail and the build is marked as failed, the MSTest build step itself does not fail. This causes problems if you have build steps after the MSTest build step which you don't want to run if you have test failures (e.g. it may not make sense to produce an installer or documentation of a build you know has bugs). The problem will hopefully be fixed in later versions of TeamCity.
If you want your build process to stop when you have test failures, you can create a new build step that uses the TeamCity REST API to detect if the current build has been marked as failed (remember that when tests fail, the build step is not marked as failed, but the build is), and then fail the current build step explicitly. Example:
Create a new Powershell build step
Script: Source code
Source code: See script below
Make sure your newly created build step comes immediately after your MSTest build step
Make sure every build step after this one has Execute step set to Only if all previous steps were successful
Script:
$xml = [xml](curl --request GET http://USERNAME:PASSWORD#HOSTNAME/httpAuth/app/rest/builds/%teamcity.build.id%)
Microsoft.PowerShell.Utility\Select-Xml $xml -XPath "/build" | % { $status = $_.Node.status }
if ($status -eq "FAILURE") {
throw "Failing build step on purpose"
}
I am new to CI and would like your thoughts and input on how to go about my problem. I would like to first start off that I have been wrestling with this for 2 days(and I don't have that much background in sys ad) so please play nice?(I am mainly a front-end web dev) :)
Basically my plan was to install jenkins then make a CI env with these steps:
poll for any changes to github
if there are, run the build script:
a. migrate the development and test dbs?(does that mean i have to put the config/database.yml in my repo?)
b. run cucumber
c. if all tests pass go to 3, else fail
run any rake setup stuff
run the server(deploy)
I have done some of the stuff by cheating:
in my local, i switch my rvm to the correct one i need(rvm use 1.8.7-p174#mygemset)
run jenkins(java -jar jenkins.war) so that it gets the RVM ruby as default
run spork in a separate terminal(because for some reason my cucumbers don't run without spork - that's another problem)
build the project manually by clicking Build
so basically, I want to automate these stuff. Maybe what I need is a set of steps to follow(general or specific, depending on your taste) so I can setup my CI up and running.
Keep in mind that my "cheats" won't do as I want to test different projects with different setups and the startup cheat just won't do. Currently, my project build was successful because all I did was to run cucumber(and all my cukes pass). I want it to be able to deploy after it passes so maybe some help there also? Thanks
Okay I will try and help you as best I can.
poll for any changes to github
This can be easily done with the Github Plugin located here
if there are, run the build script: a. migrate the development and test dbs?(does that mean i have to put the config/database.yml in my
repo?) b. run cucumber c. if all tests pass go to 3, else fail
Then all you would is run the build script you have configure in the in the build from
Select "Add Build Step" -> "Execute shell".
You can either do that which is probably what I would do because when you create build you want them to be portable so you can start up in new jenkins instances, so you dont have to setup your build machine, with build specific files.
Then you run your tests, if they fail the build should fail regardless here is some information on running ruby on rails tests. if you need to manually fail a build in a script based on a result usually exiting a script with non-zero will fail the build. If not continue and run your rake and deployment scripts.
Just a few notes on Jenkins it wont do everything for you but if you can do it manaually Jenkins can automate it. So anything you have setup running manually with a little bit of effort you can get up and running automated with Jenkins
Here is another answer you might find helpful in your general setup and ideology behind Jenkins.
Goodluck!
Is it possible to Trigger an exe to run on a failed build? Can you do this within Team City?
If you specifically want the failed builds, you can set up the dependent build as Eric said, and have that secondary buildscript use the REST API to pull up a list of the failed builds for the actual project.
If the latest build is in that failed builds list, then tell the build script to run the executable. If not, then you're all done!
http://confluence.jetbrains.net/display/TW/REST+API+Plugin
I don't think it's possible to trigger an executable to run only on failed TeamCity builds. TeamCity usually allows you to do things either always or only upon successful builds.
It would be possible to trigger an executable to run after this build is finished (failed or successful).
If that would work for you, you could set up a new build configuration that runs the executable. The new build configuration would have a "finish build" trigger. This would cause the executable to be run whenever the other build is completed.
You should add another build step with the exe you want to run and set the correct option to execute.execution options