TeamCity keeps using the same agent if it fails - continuous-integration

My current TeamCity setup has several agents with pretty much the same configuration and all of them are also eligible for running the builds. However, I have noticed that TeamCity keeps scheduling runs on the same agent in a roll, even if the build fail. Actually, it happens only when it fails. Is this a known behaviour or a bug? Is there any way to change such configuration?

Check your "Agent Requirements" in the build conf according to https://confluence.jetbrains.com/display/TCD8/Agent+Requirements

Related

TeamCity Build Agent Shows All Build Steps Incompatible

I've recently installed TeamCity 2017.2.3 (build 541047) and Octopus Deploy 2018.1.5 and originally was having success running builds and creating packages in TeamCity, but now my build agent shows all of my build steps as incompatible after a service restart.
I've uninstalled and reinstalled the buildagent, plugins (I use Node.JS build runner, Octopus Deploy integration, and xUnit) and rebuilt each of the build steps, but still run into the same problem
Build Step List
Anyone know what would be causing this issue?
Each agent should have a list of global parameters that are picked up from the OS or manually configured from the agent properties file on the machine. These are things like the path, dotnet versions, npm etc.
http(s)://<tc root>/agentDetails.html?id=<agent id>&tab=agentParameters
(You can get to the above by clicking on the agent you want to inspect)
You can then override or add to these from the root project all the way up the project tree to the build configuration.
The message you are getting is saying is that in order for the build to run it needs to have an agent with those parameters configured. Could you give a screen grab of what your build agent parameters are.
Here is an example from one of my build agents which shows some of the configuration parameters that you need.
You should not need to add these, they should be picked up automatically by the agent.
First make sure these dependencies are actually installed.
If missing install and restart the agent service (required to pick up new configuration properties).
Possibly try a machine restart in case newly installed components require a reboot.
Failing that check to see what permissions the build agent service is running under. It might not have permissions needed to poll the system but I have never seen that.

Running multiple build agents under specific user

I currently have one of my build agents in TeamCity running under a specific user due to one of the build steps. The build agent supports a couple different projects, so I want to add an additional build agent. I have successfully done so, but upon doing so the new build agent starts working, but my older build agent gets disconnected and states "Agent has unregistered." Now I am using the same user to try and run both build agents, so not sure if that is whats causing the issue or not. Both build agents are on the same machine and do have distinct names. I have tried to restart the agents in hope that it would fix itself, but have not been successful.
There are several resources covering setup of multiple build agents on the same machine, you can examine them if you wish.
Here are requirements from official documentation:
The agents are installed in separate directories.
The agents have the distinctive workDir and tempDir directories in the buildAgent.properties file.
Values for the name and ownPort properties of buildAgent.properties are unique.
No builds running on the agents have the absolute checkout directory specified.
Please, make sure that port number is changed, as this is most obvious reason why build agent could be terminated. Also, check that each build agent has its own unique name.
Otherwise, please check each build agent logs and - if it won't be clear -, share error here.

SonarQube 5.1 fails execution with message "The project already being analyzed"

We have had occurrences where a SonarQube job is running for hours, appears to be hung with no ETA on its completion in our CI environment. We have used a workaround solution of not only aborting the build job, but there seems to be a secondary process executing SonarQube on the CI agent itself. I have read that specifying sonar.forceAnalysis=true should resolve the issue but this has been deprecated as of version 3.X and there is no need to use this analysis parameter. We would like to be able to proactively address this and not have to respond after a team running SonarQube notifies us of this issue. Is there some way of specifying a timeout where the process running the analysis will just terminate?
If you are looking for killing the analysis running in your CI environment after a certain time, I would suggest you look into the feature offered by your CI environment.
There is no such thing in SonarQube.
As a side note, you should have a look into the upcoming 5.2 release of SonarQube. We have cut the connection to the database from the analyzer which means the error “The project already being analyzed” can not occur any more. In addition, quite a bunch of work done by the analysis job has been moved to the server which allow it to run faster.

Idle TeamCity agents doesn't take compatible tasks from queue

I am using TeamCity 8.1.3 and have one task in the build queue and compatible enabled agent.
In "Build Queue" teamcity says that task "Can run on" - "2 agents"
Both agents are idle.
I tried to reinstall teamcity, reinstall buildAgent and install agent on another machine.
Tried to run agent as root. I read all agent logs and didn't find any errors; only warnings for not installed VCS tools (i need only git and its installed - no warning for it).
Why wont the agent run my task?
If the VCS tools aren't installed then TeamCity can't pull down your code and run the agent.
From the TeamCity documentation http://confluence.jetbrains.com/display/TCD8/Git+(JetBrains):
Git support in TeamCity is implemented as a plugin. Git needs to be installed on the server machine and, if the agent-side checkout is used, on the agents.
and
[Build configuration] will run only on the agents where git was detected or specified in the agent properties.
So you need install git.
This page covers installation instructions for Windows, Linux and Mac: http://git-scm.com/book/en/Getting-Started-Installing-Git. This SO post discusses installation as well: Git for beginners: The definitive practical guide
I can see two possible reasons
One of the implicit requirements not fulfilled. You need to check the enqueued build info for this, it tells why exactly the build is in the queue with compatible agents but not running. Examples can be: limitation on simultaneous builds in build or template configuration, shared resources unavailable etc. This would be listed in the little popup next to the enqueued build label
I've seen this happening when a requirement wasnt fulfilled initially (I had build configuration limited to 1 build at a time) when the build was enqueued, then it was corrected but the queue cached this requirement. Re-enqueuing the build sorted that.
Hover by mouse cursor to your VCS engine (e.g. git) and maybe you'll see there this message:
Changes have not been collected yet
In that case check performance of your TC and vcs server. (this I just guess from the message)

Queue rather than run a TeamCity build while another build is running

Our busy enterprise server has 550 active build configurations running on 30 agents. We need a way to prevent some builds from running while other builds are running. I already understand artefact dependencies and this does not solve our problem as the builds do not depend on each other they just share some resources like server port numbers and database connections. Some build configurations conflict with each other when run simultaneously and we need to prevent this by having a way to queue a build (rather than run it) when one of a number of other builds are running.
We already use dumb tricks like restricting builds to specific agents, etc... I'm thinking about adding a first build step that checks for a flag in a db table or something but this would add lots of failed builds when what we really need is proper build queueing.
Am I missing something? Does this functionality already exist in TeamCity?
It looks like what you need is on its way in the form of TeamCity 8 Shared Resources. You can already get access to the EAP here.
My huge respect for 550 configurations and 30 agents.
I think you already found the solution:
Have one environment per agent
Assign configuration for specific agents
I could imagine how much it is work. But I think it's much cleaner and will work much faster than restriction to run several configurations at the same time.

Resources