I am using a jenkins configuration where the same job is being executed in different locations: one in farm1 and another in an overseas farm2.
The Jenkins master server is located in farm1.
I encounter a situation where the job on farm2 takes much more time to finish, sometimes twice the elapsed time.
Do you have an idea what could be the reason for that?
is there a continuous master-slave discussion during the build that can cause such delay?
The job is a maven junit test + ui seleniun using vnc server on the slave
Thanks in advance,
Roy
I assume your server farms have identical hardware specs?
Network differences while checking out code, downloading dependencies, etc. Workspace of Master and Slave are on different servers
If you are archiving artifacts, they are usually archived back on Master, even when the job is run on Slave.
Install Timestamper plugin, enable it, and then review the logs of both the Master and the Slave runs, and see where there is a big time difference (you can configure Timestamper to show time as increments from the start of job, this would be helpful here)
Related
So far, Our Jenkins service hanging everyday, we cannot access the web page, I have to restart Jenkins services after hanging.
In Jenkins, I only set 4 of executors. Sometime, when 4 build job are running, the CPU load almost 90%, I guess that is a reason why Jenkins dies.
So how can I find which build job consume high CPU, and how can I find a root cause why the Jenkins die, I check in system log, but did not find any useful info.
I'm running Jenkins version 2.150.1 in Ubuntu 16.04.
Thanks.
We were also facing same issues.. We did setup prometheus + graphana monitoring for Jenkins..
This video is helpful - https://www.youtube.com/watch?v=3H9eNIf9KZs
With this - we able to point the timing when CPU was hitting at its peak and then we able to findout the job causing CPU consumption. We have switched to ECS(AWS Cloud) based job runnings.. i.e CPU and memory is separately allocated to each job..
We have also added jenkins observer script on machine where Jenkins is hoasted. we have checked if jenkins is down i.e if giving 502, we restarted it using crontab script.
These things helped us to solve the problems related to jenkins downtime.
I have configured two slaves and one master machine.
After executing the performance test in Jmeter using the master slave architecture on master machine, I am getting the result log in slaves machines but I need a consolidated log summary. Please suggest me how can I get it?
Also, I need to know that how the load is distributed over the slave machines.
First question:
If you are talking about the logs (as in jmeter-server.log) there's no automatic way to collect them all. You could write a bash script or something similar to collect them from the servers.
Second question:
Every slave machine executes the full test plan completely on their own. That means that if you configure your test plan to include 10 threads, every slave will run with 10 threads giving you a total of 20.
We are migrating from CruiseControl.NET to Jenkins just to be in sync with a partner so we don't have two different CI scripts. We are trying to setup Jenkins to do something similar to what we had CruiseControl doing which was have a centralized server invoke projects (jobs in jenkins) on remote build machines.
We have multiple build machines associated to a single project so when we build the project from the centralized CI server it would invoke the projects on the remote CI servers. The remote CI servers would pull the version from the centralized CI server project.
In CruiseCruise control we setup a project that would do a forceBuild on the remote projects. The projects on the build machines used a remoteProjectLabeller to retrieve the version number so they were always in sync.
To retrieve the master build number:
<labeller type="remoteProjectLabeller">
<project>MainProject</project>
<serverUri>tcp://central-server:21234/CruiseManager.rem</serverUri>
</labeller>
To invoke the remote projects:
<forcebuild>
<project>RemoteBuildMachineA</project>
<serverUri>tcp://remote-server:21234/CruiseManager.rem</serverUri>
<integrationStatus>Success</integrationStatus>
</forcebuild>
So far in jenkins i've setup a secondary server as a slave using the java web start but I don't know how I would have the master jenkins invoke the projects setup on the slaves.
Can I setup Jenkins to invoke projects (jobs) on slaves?
Can I make the slaves pull the version number from the master?
EDIT -
Let me add some more info.
The master, and remote build machine slaves are all running Windows.
We had the central master CruiseControl kick off the remote projects at the same time so they ran concurrently and would like to have the same thing with jenkins if possible.
Jenkins has the concept of build agents, which could perhaps fit your scenario better - there's a master that triggers the build and slaves that perform it. A build can be then restricted to some categories of slaves only (e.g. if it depends on a specific software, not present on all agents). All data is managed centrally by the master, which I believe is what you are trying to achieve.
In Jenkins it is not possible to trigger a build on a slave, i.e. where a build runs is not controlled by the one who triggers it. It is controlled by the settings of the job itself. Each job has a setting called "Restrict where this job can run".
In your case you would probably have two jobs: A and B. A would be restricted to run on "master" and B would be configured to run on "slavename". Then all that is left to do is for A to trigger B.
But you had further constraints: You want A and B check out the same version from version control and you want A and B to run in parallel. There are many ways to accomplish that but the easiest is probably to define a multi-configuration job.
There is no way to turn an existing free-style job into a multi-configuration job, so you will have to make a new job.
Choose New job
Choose Build new multi-configuration project. Add a name.
Under Configuration Matrix, open the "Add axis" drop down.
Choose Slaves
Check master and the slave
Add the SCM information and build step(s)
When the job runs, it runs on both the master and the slave. Jenkins makes sure they build from the same source version.
From the /jenkins/computer url, you can add, remove, and reconfigured "nodes" which are either local or remote "build agents".
The Jobs can then be constrained to run on particular build agents, or follow various rules to select the appropriate build agent out of the available agents.
I was thinking about Jenkins too much like CruiseControl where the job is defined on the remote machine. So in Jenkins the remote projects are defined on the master and delegated to a remote machine via an agent.
I used the Java Web Start agent installed as a windows service on the remote machines. To have specific jobs run on specific remote machines I defined each remote node with a unique label in its slave configuration. To bind specific jobs to specific slaves I used the slave's label in each job configuration ("Restrict where this project can be run").
To trigger the jobs with a single master job I created a free style job that only is set to "Build other projects" and provided a comma separated list or project names. This job builds the downstream jobs in parallel.
I'm still looking for a way to send a master build number to the downstream jobs to keep them in sync always. (This is used to version DLLs and such.)
I am working on a POC currently using Jenkins as CI server. I have setup jobs based on certain lifecycle stages such as test suite and QA. I have configured these jobs to become scheduled builds based on a cron expression.
I have a request to know how to find out what the next scheduled build will be in Jenkins based on the jobs i have created. I know what was the last succesful build, the last failed but i dont know the next proposed build. Any clues!? Or is there a view plugin for this? Sorry if this is a strange request but i need to find out.
Also i need to discover if there is an issue when more than one job is running concurrently what will happen. I would have understood this is not an issue. I do not have any slaves setup, i only have the master.
Jenkins version: 1.441
I found the first issue!
https://wiki.jenkins-ci.org/display/JENKINS/Next+Executions
So can you help me on the second question please? Is there any issue with more than one job building concurrently?
Thanks,
Shane.
For the next execution date take a look at Next Execution Plugin here.
For your second question .
The number of build you can run concurrently is configurable in the jenkins server params(http:///configure : executors param).
If the number of executor is reached each new job triggered will be add in jenkins's execution queue and will be run when one running job will end
Is there a way to have my hudson slaves used by multiple hudson masters?
A bit of background info:
My build guy has set-up separate hudson masters to do the deployment and testing of our solution into different test environments. My tests are run on hudson slaves (I have 4 slaves). These slaves are associated to one specific hudson master. I want the slaves to be available for use by any of the hudson masters.
I believe the build guy chooses to use multiple hudsom masters to manage the number of jobs on each master. His set-up for one environment has 8 view tabs therefore 5 environments would mean 40 tabs. Unfortunately as is common, the solution to one problem creates another.
Yes, you can add the slaves to both Hudson masters. The problem is that each master will not be aware of the resource utilization by the other master, so you'll have to figure out some mechanism for that, such as reducing the number of executors.
Even better would be to combine the two Hudson masters into a single Hudson instance. Your question doesn't explain the motivation for having two masters.
As I cannot comment above I'll try an answer.
I think you can have several independent slaves on same machine, each attaching and discussing with its unique master. I also think that different slaves on same machine sharing same home directory is not supported, not working. And of course if they are completely independent, as Michael Donohue said above, there is a workload sharing issue to resolve.
v1.366 added support for Windows slaves running as a Win32 service to serve multiple masters
see http://hudson-ci.org/changelog.html
Hudson jobs can also be parameterized, with a default value used for scheduled jobs and a web page offered for parameter input on manually triggered jobs. That can work in some situations to reduce need for multiple jobs.
Or try the nested view plugin if number of tabs an issue and can't reduce number of jobs