TeamCity Build Agent won't upgrade - teamcity
We are trying to setup a build agent and every time we start it the log shows the following messages:
[2012-09-18 12:52:01,805] INFO - jetbrains.buildServer.AGENT - Starting agent shutdown sequence, reason: Restart agent, failed to download upgrade from server
[2012-09-18 12:52:01,821] INFO - jetbrains.buildServer.AGENT - Host configuration for downloading updates: HostConfiguration[host=http://localhost:8000]
[2012-09-18 12:52:01,821] INFO - jetbrains.buildServer.AGENT - Downloading http://localhost:8000/update/teamcity-agent.xml ==> E:\buildAgent\temp\m8a1mAwTuLIngev3yRUMPUuaYWZFmMSh
[2012-09-18 12:52:01,849] INFO - jetbrains.buildServer.AGENT - Downloading http://localhost:8000/update/plugins/agentSystemInfo.zip ==> E:\buildAgent\update\plugins\agentSystemInfo.zip
[2012-09-18 12:52:01,880] INFO - jetbrains.buildServer.AGENT - Downloading http://localhost:8000/update/plugins/amazonEC2.zip ==> E:\buildAgent\update\plugins\amazonEC2.zip
[2012-09-18 12:52:01,921] INFO - jetbrains.buildServer.AGENT - Downloading http://localhost:8000/update/plugins/ant.zip ==> E:\buildAgent\update\plugins\ant.zip
[2012-09-18 12:52:02,056] INFO - jetbrains.buildServer.AGENT - Downloading http://localhost:8000/update/plugins/antPlugin.zip ==> E:\buildAgent\update\plugins\antPlugin.zip
[2012-09-18 12:52:02,078] INFO - jetbrains.buildServer.AGENT - Downloading http://localhost:8000/update/plugins/assembly-info-patcher.zip ==> E:\buildAgent\update\plugins\assembly-info-patcher.zip
[2012-09-18 12:52:02,098] INFO - jetbrains.buildServer.AGENT - Downloading http://localhost:8000/update/plugins/clearcase-agent.zip ==> E:\buildAgent\update\plugins\clearcase-agent.zip
[2012-09-18 12:52:02,106] INFO - jetbrains.buildServer.AGENT - Downloading http://localhost:8000/update/plugins/commandLineRunner.jar ==> E:\buildAgent\update\plugins\commandLineRunner.jar
[2012-09-18 12:52:02,118] INFO - jetbrains.buildServer.AGENT - Downloading http://localhost:8000/update/plugins/coveragePlugin.zip ==> E:\buildAgent\update\plugins\coveragePlugin.zip
[2012-09-18 12:52:02,151] INFO - jetbrains.buildServer.AGENT - Downloading http://localhost:8000/update/plugins/crashDetector.zip ==> E:\buildAgent\update\plugins\crashDetector.zip
[2012-09-18 12:52:02,163] INFO - jetbrains.buildServer.AGENT - Downloading http://localhost:8000/update/plugins/cvsAgent.zip ==> E:\buildAgent\update\plugins\cvsAgent.zip
[2012-09-18 12:52:02,183] INFO - jetbrains.buildServer.AGENT - Downloading http://localhost:8000/update/plugins/dotCover.zip ==> E:\buildAgent\update\plugins\dotCover.zip
[2012-09-18 12:52:02,308] INFO - jetbrains.buildServer.AGENT - Downloading http://localhost:8000/update/plugins/dotNetPlugin.zip ==> E:\buildAgent\update\plugins\dotNetPlugin.zip
[2012-09-18 12:52:03,830] INFO - agent.impl.AgentPortFileWriter - Delete agent runtime file from E:\buildAgent\logs\buildAgent.port
[2012-09-18 12:52:03,831] INFO - jetbrains.buildServer.AGENT - Unregistering from build server: 5
Has anyone seen anything like this before? We've looked at the server logs and aren't seeing anything on them to indicate what could be wrong.
I ran into the same issue. Both the build agent and the server are installed on Windows Server 2012.
I stopped the build agent service and deleted the logs from BuildAgent\logs and restarted the build agent service so I could see a fresh log.
upgrade.log showed me that the build agent received a call from the teamcity server to upgrade. The log also showed the following:
"Please check TeamCity build agent service user have enough permissions to stop and start the service."
Using the Local Security Policy, I granted the build agent service user "logon as a service" rights but this is not sufficient to start and stop a service. By default, only members of the Administrators group can start, stop, pause, resume or restart a service. After I added the build agent service user to the Administrator group and restarted the build agent service, the upgrade finished successfully and the agent connected again.
Alternatively, as described by BatteryBackupUnit and https://web.archive.org/web/20171019005501/http://windowsitpro.com/security/letting-user-start-and-stop-services-without-granting-user-administrator-privileges, it's possible to give the TeamCity Windows user account the start, stop and pause permissions on the TeamCity Build Agent service. The article says that something like
subinacl /service spooler /grant=contoso\cortana=top
will suffice.
Or, if you don't want to use a tool that Microsoft no longer makes available, then you can use Process Explorer, as described at https://superuser.com/a/315709/12337.
I had this problem when running the agent with systemd on Ubuntu 18.04. The agent exits with code 143 when upgrading and this is interpreted as an error. It needs to be added to the list of acceptable error codes with SuccessExitStatus=143 0
Here's the full configuration:
[Unit]
Description=TeamCity Build Agent
After=network.target
[Service]
Type=forking
RemainAfterExit=yes
PIDFile=/build-agent/logs/buildAgent.pid
ExecStart=/build-agent/bin/agent.sh start
ExecStop=/build-agent/bin/agent.sh stop
User=build
Group=build
Restart=on-failure
RestartSec=5s
# agent will exit with 143 during upgrade process
SuccessExitStatus=143 0
[Install]
WantedBy=multi-user.target
The TeamCity build agent can sometimes take a long time to upgrade. If you believe that the upgrading takes too long with no positive results do as follows:
Uninstall the build agent
Download the build agent pack from your server
Reinstall the agent
Upgrading process should go faster after this. If it does not help it is better to contact JetBrains tech support.
A really simple way of doing this if your enterprise security policy doesn't allow you to add users to the 'Administrators' group, but the user your logged on as does have elevated rights. Stop the service running, via the Services.msc and then open a command prompt (with elevated rights) in the agent bin directory and run
agent.bat start
Let the agent update with this user and then once it has finished and you can see it successfully registered in the Teamcity UI. Kill the processes and restart the Service.
If you're using a "jailed" build user like we are, you'll see "Please check TeamCity build agent service user have enough permissions to stop and start the service."
There are workarounds but no graceful solution to this. Here's why:
Workaround 1: "TCBuildAgent" service needs "jailed" needs start/stop service credential access assigned to it. SubinACL is your best tool for assigning this (lengthy but full discussion).
TeamCity removes "TCBuildAgent" and installs a new one, effectively eliminating the service credentials assigned to it. So, you must issue SubinACL grant access commands every time you upgrade. Clumsy and annoying
Workaround 2: Aforementioned tip of adding "jailed" to Administrators group is viable and permits a proper upgrade, though violates the concept of a "jailed" user.
After much fiddling with SubinACL, I've thrown in the towel and just temporarily add "jailed" to Administrators group for the upgrade process then remove it afterward.
I trust that Jetbrains isn't gonna do something awful and malicious with TCBuildAgent... not during upgrade, anyway ;)
I had a similar problem. helped me to reinstall the antivirus.
http://devnet.jetbrains.com/thread/440728
Stop the agent:
./agent.bat stop
Reinstall agent.
Start the agent:
./agent.bat start
Related
Jenkins agent as Windows service starts and stops
I am attempting to install a Jenkins agent as a Windows 10 service. I install the service with jenkins-slave.exe install and have configured the associated xml file, and the service appears to be properly installed. However, when I start the service, it stops immediately. A jenkins-slave.wrapper.log file is generated, containing 2021-09-09 11:30:53,503 DEBUG - Starting WinSW in the CLI mode 2021-09-09 11:30:54,191 INFO - Installing the service with id 'JenkinsAgent' 2021-09-09 11:30:54,238 DEBUG - Completed. Exit code is 0 2021-09-09 11:31:02,164 DEBUG - Starting WinSW in the service mode 2021-09-09 11:31:02,211 INFO - Downloading: https://<jenkins-server>/jnlpJars/slave.jar to C:\jenkins-agent\slave.jar. failOnError=False 2021-09-09 11:31:02,321 INFO - Skipped downloading unmodified resource 'https://<jenkins-server>/jnlpJars/slave.jar' 2021-09-09 11:31:02,321 INFO - Starting java.exe -Xrs -jar "C:\jenkins-agent\slave.jar" -jnlpUrl https:/<jenkins-server>/computer/jetdamdev2/jenkins-agent.jnlp -secret <secret> -workDir "C:\jenkins-agent" 2021-09-09 11:31:02,336 DEBUG - Completed. Exit code is 0 This seems to indicate that the java process is started, but exits about 115ms later. If I copy the java-launch line into a command-line terminal, it successfully connects to the Jenkins master and stays running. I have been chasing this issue for quite a while now, with no success. Can someone please tell me what I have done wrong with the service and help me fix this?
I know I am late for the party, but maybe it comes handy for others: I was following this article: https://hayato-iriumi.net/2019/05/23/how-to-install-jenkins-slave-as-windows-service/ I have faced the same problem with WinSW, and ended up wrapping the whole agent startup command line in a batch file, and run the batch from WinSW. So my jenkins-slave.xml looks like this: <service> <id>JenkinsSlave</id> <name>Jenkins agent</name> <description>This service runs an agent for Jenkins automation server.</description> <executable>c:\jenkins\agent.bat</executable> <logmode>rotate</logmode> </service> And agent.bat like this: java -jar agent.jar -jnlpUrl http://<jenkins master URL>/jenkins-agent.jnlp -secret <secret> -workDir "c:\jenkins" For me this solved the problem. I am stuck at an older, 2.319.1 version of Jenkins for legacy reasons.
CDH cluster installation failing in "distributing" stage- failed due to stall on seeded torrent
Hi, We are trying to install CDH cluster on Redhat 7 remote server using cloudera-installer.bin file, in standalone mode( we have only 1 host) . We are specifying hostname/ip address of the machine during installation , it is able to resolve it. But the installation halts during parcel distribution stage. Here are the logs of cloudera-scm-agent :(We tried both cloudera express edition and entrerprise trial version too) ['http://INHUSZ1-V250152:7180/cmf/parcel/download/CDH-5.15.1-1.cdh5.15.1.p0.4-el7.parcel'] location=/opt/cloudera/parcels/.flood/CDH-5.15.1-1.cdh5.15.1.p0.4-el7.parcel progress=0] [03/Oct/2018 10:11:55 +0000] 28315 Thread-13 downloader INFO Current state: CDH-5.15.1-1.cdh5.15.1.p0.4-el7.parcel [totalDownloaded=0 totalSize=2120090032 upload=0 state=downloading seed=['http://INHUSZ1-V250152:7180/cmf/parcel/download/CDH-5.15.1-1.cdh5.15.1.p0.4-el7.parcel'] location=/opt/cloudera/parcels/.flood/CDH-5.15.1-1.cdh5.15.1.p0.4-el7.parcel progress=0] [03/Oct/2018 10:11:57 +0000] 28315 Thread-13 downloader INFO Current state: CDH-5.15.1-1.cdh5.15.1.p0.4-el7.parcel [totalDownloaded=0 totalSize=2120090032 upload=0 state=downloading seed=['http://INHUSZ1-V250152:7180/cmf/parcel/download/CDH-5.15.1-1.cdh5.15.1.p0.4-el7.parcel'] location=/opt/cloudera/parcels/.flood/CDH-5.15.1-1.cdh5.15.1.p0.4-el7.parcel progress=0] [03/Oct/2018 10:11:59 +0000] 28315 Thread-13 downloader INFO Current state: CDH-5.15.1-1.cdh5.15.1.p0.4-el7.parcel [totalDownloaded=0 totalSize=2120090032 upload=0 state=downloading seed=['http://INHUSZ1-V250152:7180/cmf/parcel/download/CDH-5.15.1-1.cdh5.15.1.p0.4-el7.parcel'] location=/opt/cloudera/parcels/.flood/CDH-5.15.1-1.cdh5.15.1.p0.4-el7.parcel progress=0] Please let us know what can be done
I just had the same error message and stall during install at parcel distribution stage. Installing a single node (test) cluster on CentOS 7.5 with CDH Express 5.15. Solution that worked for me was adding the node IP and FQDN to /etc/hosts (previously it only contained entries for 127.0.0.1 localhost): [root#mynode ~]# vi /etc/hosts 192.168.1.1 myhostname.mydomain Then restarted Cloudera SCM Agent: [root#mynode ~]# service cloudera-scm-agent restart Installation then continued successfully.
Do the following: Stop all services. Deactivate all in-use parcels. Shut down the Cloudera Manager Agent on all hosts. Move the existing parcels to the new location. Configure the host parcel directory. Start the Cloudera Manager Agents. Activate the parcels. Start all services.
Delete the corresponding parcels package from below folder including .torrent file /opt/cloudera/parcels/.flood/ Download and distribute This is happening because .torrent file is corrupted
RabbitMq Installation issue
I am trying download and create a queque but I have problem. This is my list what I am doing: I download erlang http://www.erlang.org/downloads (for windows x64) I download rabbitmq https://www.rabbitmq.com/install-windows.html (for windows ) I install erlang and next install rabbimq I write rabbitmq-plugins.bat enable rabbitmq_management in console and I have: C:\Program Files\RabbitMQ Server\rabbitmq_server-3.6.0\sbin>rabbitmq-plugins.bat enable rabbitmq_management Plugin configuration unchanged. Applying plugin configuration to rabbit#----... failed. - Could not contact node rabbit#----. Changes will take effect at broker restart. - Options: --online - fail if broker cannot be contacted. --offline - do not try to contact broker. So i write in consol "sc query rabbitmq" and I have : [SC] EnumQueryServicesStatus:OpenService FAILED 1060: The specified service does not exist as an installed service. I know that rabbit is started by the process C:\Program Files\erl7.0\erts-7.0\bin\erlsrv.exe but I have this process so erlang was install correctly. I find people who have similar problem but they only "restart the RabbitMQ service and then try again" but I can't do this because my service does not exist. Please give me some advice.
Ok I know. In console I am writing: rabbitmq-plugins.bat enable rabbitmq_management rabbitmq-server start and RabbitMQ work.
TeamCity build agent disconnected
I installed a build agent on Windows 7 and always have the agent show up as "disconnected". The log teamcity-agent.txt shows below information If this is the first time this agent registered on the server make sure it is authorized by administrator in the server web UI. [2012-09-05 02:56:14,416] INFO - jetbrains.buildServer.AGENT - Agent registration finished. [2012-09-05 02:56:14,418] INFO - jetbrains.buildServer.AGENT - Stop command was not performed. No build to stop with reason: BuildInterruptReason.AGENT_SHUTDOWN [2012-09-05 02:56:14,418] INFO - jetbrains.buildServer.AGENT - Starting agent shutdown sequence, reason: Restart agent, failed to download upgrade from server [2012-09-05 02:56:14,424] INFO - jetbrains.buildServer.AGENT - Host configuration for downloading updates: HostConfiguration[host=http://myteamcity:8080] [2012-09-05 02:56:14,424] INFO - jetbrains.buildServer.AGENT - Downloading http://myteamcity:8080/update/teamcity-agent.xml ==> C:\BuildAgent\temp\iQ0fjie6zR125neulJeqiYauT8zQ5zOI [2012-09-05 02:56:15,070] INFO - jetbrains.buildServer.AGENT - Downloading http://myteamcity:8080/update/plugins/agentSystemInfo.zip ==> C:\BuildAgent\update\plugins\agentSystemInfo.zip [2012-09-05 02:56:15,747] INFO - jetbrains.buildServer.AGENT - Downloading http://myteamcity:8080/update/plugins/amazonEC2.zip ==> C:\BuildAgent\update\plugins\amazonEC2.zip [2012-09-05 02:56:16,617] INFO - agent.impl.AgentPortFileWriter - Delete agent runtime file from C:\BuildAgent\logs\buildAgent.port [2012-09-05 02:56:16,619] INFO - jetbrains.buildServer.AGENT - Unregistering from build server: 18 [2012-09-05 02:56:16,800] INFO - jetbrains.buildServer.AGENT - Shutdown agent WebServer start [2012-09-05 02:56:16,801] INFO - jetbrains.buildServer.AGENT - Shutdown agent WebServer finish [2012-09-05 02:56:16,801] INFO - jetbrains.buildServer.AGENT - Shutdown agent finish [2012-09-05 02:56:16,968] INFO - jetbrains.buildServer.AGENT - Downloading http://myteamcity:8080/update/plugins/ant.zip ==> C:\BuildAgent\update\plugins\ant.zip [2012-09-05 02:56:50,086] INFO - jetbrains.buildServer.AGENT - Downloading http://myteamcity:8080/update/plugins/antPlugin.zip ==> C:\BuildAgent\update\plugins\antPlugin.zip [2012-09-05 02:56:52,070] INFO - jetbrains.buildServer.AGENT - Downloading http://myteamcity:8080/update/plugins/assembly-info-patcher.zip ==> C:\BuildAgent\update\plugins\assembly-info-patcher.zip [2012-09-05 02:56:53,464] INFO - jetbrains.buildServer.AGENT - Downloading http://myteamcity:8080/update/plugins/clearcase-agent.zip ==> C:\BuildAgent\update\plugins\clearcase-agent.zip [2012-09-05 02:56:55,375] INFO - jetbrains.buildServer.AGENT - Downloading http://myteamcity:8080/update/plugins/cmake-runner.zip ==> C:\BuildAgent\update\plugins\cmake-runner.zip [2012-09-05 02:56:57,190] INFO - jetbrains.buildServer.AGENT - Downloading http://myteamcity:8080/update/plugins/commandLineRunner.jar ==> C:\BuildAgent\update\plugins\commandLineRunner.jar [2012-09-05 02:56:58,218] INFO - jetbrains.buildServer.AGENT - Downloading http://myteamcity:8080/update/plugins/coveragePlugin.zip ==> C:\BuildAgent\update\plugins\coveragePlugin.zip [2012-09-05 02:57:11,629] INFO - jetbrains.buildServer.AGENT - Downloading http://myteamcity:8080/update/plugins/crashDetector.zip ==> C:\BuildAgent\update\plugins\crashDetector.zip [2012-09-05 02:57:12,436] INFO - jetbrains.buildServer.AGENT - Downloading http://myteamcity:8080/update/plugins/cvsAgent.zip ==> C:\BuildAgent\update\plugins\cvsAgent.zip [2012-09-05 02:57:20,720] INFO - jetbrains.buildServer.AGENT - Downloading http://myteamcity:8080/update/plugins/dotCover.zip ==> C:\BuildAgent\update\plugins\dotCover.zip [2012-09-05 02:58:00,011] INFO - jetbrains.buildServer.AGENT - Downloading http://myteamcity:8080/update/plugins/dotNetPlugin.zip ==> C:\BuildAgent\update\plugins\dotNetPlugin.zip How can I get the agent connected?
Here is a very comprehensive blog on re-connecting disconnected agents http://michaelbarnesjr.wordpress.com/2012/10/18/teamcity-build-agent-disconnected/ Here are the steps (copied from the above link) If an agent becomes disconnected for whatever reason, you can follow these steps to re-connect it: ssh to the server that the disconnected agent is on. go to the build agent’s bin directory (i.e. /opt/teamcity/buildagent/buildagent01/bin/) su to the user that the agent runs as (i.e. appBuilder) run agent.sh start That will start the agent, and after a few seconds or so, the next time you go to TeamCity, it will be connected. If Team City is running on a windows server, you can connect to that server, go to Services and restart the build agent that has become disconnected.
In my case the agent was started but listed as disconnected, after waiting several minutes it was still disconnect. So I went to the bin directory on the build agent like Peter suggested and ran the following two commands ./agent.sh stop ./agent.sh start essentially just restarting the build agent. That did the trick for me.
First, make sure in the TeamCity website dashboard to authorize the agent if you haven't done so already. Next, make sure the agents "ownPort" is not blocked by a firewall on the agent. This is the port that TeamCity will use to send messages to the agent. The value can be defined in "${agent.home}/conf/buildAgent.properties". As of TeamCity 2017.2.1, the "ownPort" is not present, nor is there any embedded documentation (i.e. commented out examples). The default value is 8080, but you can just add the property and value you want like this: ## Defines which port the agent will use to receive messages from TeamCity. ## Defaults to 8080 if not present. ownPort=9091
In my case under Windows 8 Professional: TC build agent upgrade failed I checked out BuildAgent\logs\teamcity-agent.log, there was trouble with the "upgrade process". I checked out BuildAgent\logs\upgrade.log, there were more detailed "DEBUG" messages. The problem was "access denied" on shutting down service. I added the user account of the TC server service to "Administrators" to let it stop the agent service. I removed the user account of the TC server service from "Administrators" user group. Restarting both service (TC server + Agent) led finally to have: BuildAgent\logs\buildAgent.port file: the agent was finally connected after successfull.
I had issues with the server and change the port for the server. So, the agent was unable to connect to the server since I only changed the server port and not the agent properties. Once I updated the agent properties, the agent started working again. The properties are found under: TeamCity\buildAgent\conf\buildAgent.properties The property is found like: serverUrl=http://localhost:8080
Starting the TeamCity Agent manually in the Services panel resolved the issue for me (on a Windows machine). Problem was indicated in the upgrade.log (in BuildAgent\logs directory): ERROR - r.agent.upgrade.UpgradeRunBase - Failed to start agent after upgrade: java.io.IOException: Failed to start TeamCity build agent service. Please check TeamCity build agent service user have enough permissions to stop and start the service.
I ran into this problem when working remotely over VPN and my IP address changed. In the conf/buildAgent.properties file, I changed my assigned IP for the serverURL property to use localhost instead, and the agent connected right up. ## The address of the TeamCity server. The same as is used to open TeamCity web interface in the browser. #serverUrl=http://111.222.333.444:8111 serverUrl=http://localhost:8111
I faced this issue on Windows 10, when I changed the port of TeamCity from "8111" to "80". To fix this issue, I opened the file C:\TeamCity\buildAgent\conf\buildAgent.properties and chenged the first line from serverUrl=http\://localhost\:8111 to serverUrl=http\://localhost\:80.
TeamCity build agent continually "Unable to start upgrade: Connection reset"
My TeamCity agent is contniually trying to download upgrades from the TeamCity server. It always downloads the files, then fails to start the installation, with the error: WARN - jetbrains.buildServer.AGENT - Unable to start upgrade: Connection reset My server is running at http://localhost:9912 How do I stop this endless cycle? Latest version of TeamCity downloaded from JetBrains
I'm guessing you're running the agent as an AD account rather than under the machine account. Run the agent service under an account with administrative permissions for the upgrade. I don't remember the specifics of what it's trying to write but it won't if it's a standard user.