On Jenkins it is possible to set same arguments per agent but with different values e.g
Mac agent:
JENKINS_REPOS=/Users/johndoe/Repositories
UNITY_VERSIONS_PATH=/Users/johndoe/Editors
Windows agent:
JENKINS_REPOS=C:\Program Files\android\Repositories
UNITY_VERSIONS_PATH=C:\Program Files\android\Editors
once done, I can run the same pipeline on both agents and values change dynamically, according to agent I specify before build starts e.g agent Mac
I saw there is an option to define arguments on agents, however it does not really work except for system.agent.checkoutDir - I can set custom directory for each build.
How can I do it in same way on Teamcity? Thanks in advance.
Related
I am writing a release pipeline to upload an apk (installer) to Google Play. I am using the Google Play - Release task to do this. This is in a classic pipeline (our code is in TFS.)
One of the options is the Track to upload the apk to. The options are:
I want to set this option based on a variable that is set in one of the previous tasks. I have a previous task that sets the release.task variable to either Internal test or Production based on whether it is a public release or not. I am using it in the Google Play task like this:
However when I run the pipeline it does not recognise the value, even though it is one of the valid options:
Is there a way to get around this? I need to control which track the pipeline writes to based on a value in our code base.
I've found the issue with this scenario - you can set the value from a variable but the values are different (for this task) from those on display in the UI. For the two tracks I am using the values are:
Production -> production (note the difference in the case)
Internal test -> internal
We are having issues setting the Jenkins environment variables on our dynamic EC2-Fleet.
We already have a fixed master (linux) and a fixed Windows slave but wanted to add slaves dynamically when the load on the system becomes heavy.
For this we created a Spot Request Instance in AWS spinning up linux machines from an AMI and control this via the EC2-fleet-plugin in Jenkins.
Before this EC2-fleet can be of any help, our jobs must be able to run on its nodes.
Most of our jobs use Jenkinsfiles and need certain environment variables to be set but the EC2-fleet-plugin does not provide the possibility to set environment variables (https://issues.jenkins-ci.org/browse/JENKINS-36544).
As suggested on this ticket (JENKINS-36544), we tried to set the environment variables in "System Configuration" for the dynamic ec2 slaves and set the environment variables for the other nodes on the "Node Configuration" overriding the "System Configuration", or so we thought.
This should work if this bug wouldn't exist: https://issues.jenkins-ci.org/browse/JENKINS-44425. Because of this bug the "System Configuration" overrides the "Node configuration" instead of vice versa. So we can't use this as the existing nodes would not have the correct environment variables anymore.
As a last resort we tried to set the environment variables on the dynamic ec2 slaves by creating an /etc/profile.d/jenkinsvars.sh on the AMI used by the Spot Request Instance.
This script would be automatically run on login system wide (https://help.ubuntu.com/community/EnvironmentVariables#A.2Fetc.2Fprofile.d.2F.2A.sh).
Next to that we also attempted to set them in /home/ubuntu/.profile on the AMI singling out the ubuntu user which is the user running the Jenkins agent (https://help.ubuntu.com/community/EnvironmentVariables#A.2BAH4-.2F.profile).
But it appears Jenkins does not use these environment variables but its own...
A way that works is to adapt the jobs to load a groovy file that's part of the AMI to set the environment variables we need but that would mean to change almost all jobs we have, next to all Jenkinsfiles that are included in our repositories (Bitbucket project).
We would like to avoid this....
Try the following strategy:
Leverage User Data to run a shell script when the Spot Instance launches. It is the primary recommended way by Amazon and the plugin authors.
Instead of saving variables into the environment, have the user data script save them into either /var/tmp or /etc/profile or in parameter store. Refer to answers in this SO question. If you want to encrypt your info use KMS parameter store, if you dont care use one of the others.Choose one of the answers to best fit your needs.
Alter your Jenkins job to pause until your user data script has completed running (refer to the documentation from the plugin)
Change your Jenkins job to pick up the variables from the location you chose in step 2.
Try restarting the server environment.
Just saying.
So we can't use this as the existing nodes would not have the correct environment variables anymore.
Update your existing nodes to load the environment variables when they are provisioned / started, then remove them from the System configuration, then add them to the Node configuration.
You could also try setting the Slave command prefix field to ENV_VAR1=val1 ENV_VAR2=val2, although I haven't tried that.
Thirdly, you can try putting your variables directly into /etc/profile which should always be loaded no matter what user you are logging in as.
However, the easiest by far is to make all of your drones/agents exactly the same and set your environment variables in whatever scripts you run to build your projects. Use docker to pull dependencies to the agents as necessary during the job and to set up specific environments for your applications. This greatly simplifies the maintenance and configuration of your agents.
The Jenkins version or version of the EC2 plugin are missing in the question, but according to the description in this merged pull request, this bug should be fixed now: https://github.com/jenkinsci/ec2-plugin/pull/440#issuecomment-597160730
Jenkins version:
so this change works in both <=2.204, and >=2.205
EC2 plugin version: >=ec2-1.50
JENKINS-36544 - Fix Node Properties on Jenkins 2.205+ (#440) #jhansche
From the Pull Request description:
Navigate to the cloud configuration screen (this moves to a new page >=2.205)
Click "Add a new cloud"
Click "Amazon EC2"
Under the "AMIs" section, click "Add"
At the bottom of the AMI block, expand "Advanced"
Expect to see "Node Properties" block at the bottom of the block
Node Properties has the Environment variables.
I would like my build job to not build on specific teamcity agents and there is no common pattern for the build agent names. Is there a way I can isolate multiple agents from getting picked up by a single job.
For example I can make sure that the build job does not pick up 1 agent using the following parameter.
teamcity.agent.name
does not match
agent-001
How can I similarly tell the teamcity job to not run on the following agents as well.
"123-agent"
"my_agent"
"test_agent"
"agent_do_not_use"
I cannot use the same parameter, teamcity.agent.name with does not match for multiple agents.
Can you all teamcity experts help me out here please on what is the best way to achieve this.
You can add agent requirement with "does not match" condition which accepts regular expression and set it to:
123-agent|my_agent|test_agent|agent_do_not_use
Using an agent requirement based on presence (or absence) of a specific property coming from agent's buildAgent.properties file would probably be a better solution to using agent names in the requirement.
Alternative means to manage agent's compatibility are: use agent pools and use agent's Compatible Configurations set to a limited set.
You can add a specific parameter inside the agent configuration on the local machine inside: C:\BuildAgent\conf\buildAgent.properties
Then, you can add something specific like: system.Is<MyFeature>Available=True
Then, in teamcity configuration, you will add an Agent Requirement with this parameter.
We're using TeamCity 9 and have a pool of agents, only some of which can run a particular job we've created. It's going to be a while before a new pool can be created with new agents, and in the meantime we would like to set this job to run only on specific ones.
In the configuration settings we can choose system.agent.name, but that only allows us one agent. Is there a way to configure things so that I can run on multiple agents based on their name, so that, say teamcity_agent01, teamcity_agent07 and teamcity_agent12 are eligible?
If the capabilities you want on the agent expose environment variables you can have a command line step refer to the environment variable and that will scope the job to a suitable agent.
e.g.
echo "%env.PATH_TO_GIT%"
You can add environment variables to your agents like this
open file “/conf/buildAgent.properties” at agent
system. This file contains parameters in key=value format.
To add a system properties add line like -> system.os=Linux
To add an environment variable add line like ->
env.JAVA_HOME=/home/java/jdk1.6.0_13 the
The beauty of this is that as you add more capabilities and agents then you don't need to revisit each job to change where it can run
For Teamcity 9.x
Next to the Projects, and Changes select Agents.
Select the agent that you want to run specific jobs on.
Select the "Compatible Configurations" tab.
On the "Current run configuration policy" dropdown select "Run assgined configurations only"
Click on the "+ Assign configuration" button.
Compatible configurations are colored blue. Select as you wish.
The timing of the configurations and when they should run can be configured under each build configuration triggers tab.
I have a TeamCity project with multiple sub-probjects. How do I get a subproject to run on a specific agent without hard coding it in the agent requirement upfront? I want to do something like..
That way instead of copying the subproject for each agent I can just have one project and specify an agent name at execution time in some config. Using TeamCity 9.
You can create agent pools and assign specific agents to a project. To do this click on the "agents" link at the top .
Basically, you can define that targets of a particular project should only run on particular agent.
EDIT
If you want to define it at runtime, you have to go back to defining an agent requirement in the targets. You set a paremeter for agent name in the first target .In all the subsequent targets, you can create build chains and pass parameters from one target to another. In the definition of the target, you set an agent condition saying agent name matches whatever value you are passing in your custom variable In the first target you can have a prompt/dropdown and you can initialize an agent based run for a complete build chain.
Or you can assign and unassign agents from the projects pools before each run and then run the targets. For ex in agent pool you can set Project A to only build on agent 1 . After the run is complete, you change Project A to run only on target2 .
I will recommend the first option over the second
This plugin may do what you want. You can define a series of different values for the same parameter(s) and it will run across all of them. This could be agent names or environment variables or node.js versions or whatever.
https://github.com/presidentio/teamcity-matrix-build-plugin