We just started using Jenkins for continuous intergration. The code is pulled from Perforce. We have one jenkins master (Windows VM) and 3 slaves (Windows VMs). I am more the VMware admin than a programmer.
I have been trying to tweak more and more the Jenkins slave setup. Now they are configured as 16vCPUs + 48GB of RAM per slave. Each time during a build, the CPU is always spiking at 100%. We are closing the build in 2h20m but the goal is to reach 1hr.
What is the best way to do so? What type of tweaks in VMware? How can we push through the build faster?
Thanks!
Try to make it parallel if possible...
Build Pipeline Plugin
The beauty of this is that you can trigger parametrized or regular.
So your master can start the process and the three slaves will make sure isolated steps are accomplished.
This may give a better view.
Related
Currently we are running some of our small ruby scripts through CI machines like teamcity. But the problem is that teamcity is only free to a certain point and we are reaching that cap. The thing I like about teamcity is the fact that I can define how to run the scripts in it and then have the logs shown in each of the "build" processes so if something goes wrong or I want to verify something I don't have log onto the server and inspect individual files.
The problem is that I need to be able to run the same process at about 4x the capacity I am now, which means I need about 4 times the build agents which leaving the free licensing. Now obviously I could just spin up more teamcity servers but that then becomes a pain.
So my question is, what is another way that I could just basically setup cron processes on linux machines (i have a lot "freely") but then give myself the exposure and ease of access of logs similarly that I gain from teamcity. Obviously I know that setting up cron processes aren't hard but I really want to avoid having to log onto the machine to check and see if my automated processes are running correctly or struggling.
Thanks in advance!
p.s. I also have access to windows machines if there is an easier way to do it there.
Install an open-source CI server like Jenkins if you want to host it yourself. You can also run it on your own machine, though it's usually better to have it always on in the cloud.
On one of my server's teamcity is running all build steps extremely slowly.
Using Teamcity perfmon I traced it to 100% disk usage doing simple tasks like running a nant script with a copy.
How do I fix this?
It turned out there was a type of Windows logging switched on on the server. Teamcity does a lot of things, and so the logging of what teamcity was doing was disk heavy. The type of logging was identified by looking at what process was hogging disk usage using Task Manager. I can't remember what it was, but switching off this Windows related logging solved the problem
I'm trying to introduce continuous integration in an old project, and we've got quite specific situation - it's possible to put the CI server only on our test server that runs on CentOS. The server has quite a lot of unused RAM and CPU capability.
However, we need to run Ant builds on Windows (this also used to be how the project did packaging before), however it turned out that not the same output (after binary compare) is produced by just using Unix versions of Java and Ant.
I drew up a diagram of how in my mind it could work, but I'm really wondering whether that is even possible (with already given tools).
The black part is implemented, I'm curious whether the red part could be possible. Could the Jenkins slave communicate with master on different OS?
It should be possible. I have a feeling you will need to play with your network settings. But if before you start changing anything see if you can start a headless slave by following these directions: https://wiki.jenkins-ci.org/display/JENKINS/Step+by+step+guide+to+set+up+master+and+slave+machine
Using VirtualBox for CentOS, it will possible to run a Windows VM on your CentOS host.
I'm not sure you need Docker to launch your Jenkins slave.
It maybe better to use a standard JNLP Windows service to connect your Windows slave to Dockerised Jenkins master.
If the master is not able to view the Windows node using this method, you may have to tweak your network configuration on the Windows VM.
But I'm not sure it's necessary.
I would like to run Teamcity (with a build agent) in a Linux VM to handle our none-.net projects. But in the same breath I'd like to have a BuildAgent setup on a Windows server to handle all of the .net projects.
I can't think of any reasons why this wouldn't work but has anyone any experience and any ideas about the problems I might encounter before I spend too much real time on this?
Ta
It's fully supported. TeamCity also knows which agents to route builds to.
This is a very normal scenario and many project I know do this without any problems. Just make sure that for the builds' Agent Requirements, you properly direct the appropriate job to the appropriate agent. One criterion can be that agent.os.name should contain Windows or Linux etc.
I was wondering if anybody out there is doing their software builds using something like Amazon's EC2. I was thinking about trying to move our builds into that environment. Right now our builds are serial but only because we don't have enough computers to run all the components in parallel. Using EC2 we could create 50 or so computers, run them for a few minutes in massive parallel and then send the build results back to our site. Once we're done we could shutdown or destroy the machines. This would save us a bunch of time since the bottle neck is really the builds and not the size of the results.
Is anybody else doing this? Can you offer any advice?
My company runs our build system on EC2; we have a much smaller setup than the one you're talking about, but we have a build controller instance running Hudson which kicks off builds on a separate, clean instance and then distributes the build artifacts to our repository server (which also happens to be on EC2)
Using a cloud solution is ideal for what you're describing, since you can spin up the build servers only when you need them and be confident of building from a fixed baseline each time. The only downside I can think of is the build time; an EC2 instance can take up to 10 minutes to start up, so you either have to add that on to your total build time or keep the build servers running continuously.