I am using Jenkins to create nightly builds and deploy them to my maven repository. In order to reduce the daily bandwidth for developers I want to change the deployment logic so that it's only deployed if changes with reference to our latest deployment are detected.
I found some plugins that seem to be suitable, but since I'm not too involved into the Jenkins use-ability I wanted to ask if there is an "easy" way to implement this?
I thought about some simple test before executing the deployment process:
Check if there are changes made within the project code or the dependencies
=> NO: then nothing should be deployed
=> YES: deploy the new version
I just stumpled over the Conditional BuildStep Plugin. But I'm not sure if this plugin fits best for our approach. I don't want to mess up the whole configuration.
FYI: I am using Jenkins 1.608, Tortoise SVN, and the deploy-Plugin 1.1
Every answer and help is highly appreciated!
Setup your job to run nightly (current configuration)
Setup a conditional build step (shell or batch, depends on your OS) that will check the revision number in repository and compare to current revision number in workspace.
If not higher, quit the build.
If higher, continue build as usual
Edit:
To avoid the build even being triggered, here is a theoretical solution.
Install Poll SCM Now plugin, this will add a "poll now" button to the job, that polls SCM for changes. If changes found, a build is triggered, if not, nothing happens.
Configure the polling permissions to anonymous users (else you'd need to implement login/token in next steps)
Configure the job to poll SCM infrequently (like once a year)
Configure a cron job (or a scheduled task in Windows) for a nightly schedule
Have that cron/task do a curl or wget to http://JENKINS_URL/JOB_URL/poll
Related
I have five TeamCity builds that are triggered to run at 01:00. Since they all run on the same Agent, in effect they are queued to run one after the other. Each build takes between 10-60 minutes to complete.
What I'd like to guarantee is that all five builds are run on the same revision. Currently this is not guaranteed because in case a contributing developer can't fall asleep and decides to commit something at 01:30, all builds that start running after that time will run on a different revision.
My question:
Is it possible to configure a build to use a specific VCS revision from a specific time? E.g., configure a build to use the "latest revision at the time of 01:00 today"?
Any other suggestions on how to deal with this problem would be highly appreciated as well.
One of the builds might be set up to have the others as snapshot dependencies. When that build is triggered (e.g. by schedule trigger), its dependencies will be triggered having VCS revision fixed.
Or even new build configuration might be added, it will do nothing except to triggering its snapshot dependencies (your existing configurations).
You can use independent build(or one of a builds will be a master) configuration with trigger at time you need. New config will starts other configs by PS script which calling REST API "Triggering a Build". You can point specified change for all builds(can be get with API "Get pending changes for a build configuration"). This will guarantee all builds will start with same revision. This way might help if dependencies are unacceptable.
we're trying to improve our Jenkins setup. So far we have two directories: /plugins and /tests.
Our project is a multi-module project of Eclipse Plugins. The test plugins in the /tests folder are fragment projects dependent on their corresponding productive code plugins in /plugins.
Until now, we had just one Jenkins job which checked out both /plugins and /tests, built all of them and produced the Surefire results etc.
We're now thinking about splitting the project into smaller jobs corresponding to features we provide. It seems that the way we tried to do it is suboptimal.
We tried the following:
We created a job for the core feature. This job checks out the whole /plugins and /tests directories and only builds the plugins the feature is comprised of. This job has a separate pom.xml which defines the core artifact and tells about the modules contained in the feature.
We created a separate job for the tests that should be run on the feature plugins. This job uses the cloned workspace from the core job. This job is to be run after the core feature is built.
I somehow think this is less than optimal.
For instance, only the core job can update the checked out files. If only the tests are updated, the core feature does not need to be built again, but it will be.
As soon as I have a feature which is dependent on the core feature, this feature would either need to use a clone of the core feature workspace or check out its own copy of /plugins and /tests, which would lead to bloat.
Using a cloned workspace, I can't update my sources. So when I have a feature depending on another feature, I can only do the job if the core feature is updated and built.
I think I'm missing some basic stuff here. Can someone help? There definitely is an easier way for this.
EDIT: I'll try to formulate what I think would ideally happen if everything works:
check if the feature components have changed (i.e. an update on them is possible)
if changed, build the feature
Build the dependent features, if necessary (i.e. check ob corresponding job)
Build the feature itself
if build successful, start feature test job
let me see the results of the test job in the feature job
Finally, the project job should
do a nightly build
check out all sources from /plugins and /tests
build all, test all, send results to Sonar
Additionally, it would be neat if the nightly build was unnecessary because the builds and test results of the projects' features would be combined in the project job results.
Is something like this possible?
Starting from the end of the question. I would keep a separate nightly job that does a clean check-out (gets rid of any generated stuff before check-out), builds everything from scratch, and runs all tests. If you aren't doing a clean build, you can't guarantee that what is checked into your repository really builds.
check if the feature components have changed (i.e. an update on them is possible)
if changed, build the feature
Build the dependent features, if necessary (i.e. check ob corresponding job)
Build the feature itself
if build successful, start feature test job
let me see the results of the test job in the feature job
[I am assuming that by "dependent features" in 1 you mean the things needed by the "feature" in 2.]
To do this, I would say that you have multiple jobs.
a job for every individual feature and every dependent feature that simply builds that feature. The jobs should be started by SCM changes for the (dependent) feature.
I wouldn't keep separate test jobs from compile jobs. It allows the possibility that successfully compiled code is never tested. Instead, I would rely on the fact that wen a build step fails in Jenkins, it normally aborts further build steps.
The trick is going to be in how you thread all of these together.
Let's say we have a feature and it's build job called F1 that is built on 2 dependent features DF1.1 and DF1.2, each with their own build jobs.
Both DF1.1 and DF1.2 should be configured to trigger the build of F1.
F1 should be configured to get the artifacts it needs from the latest successful DF1.1 and DF1.2 builds. Unfortunately, the very nice "Clone SCM" plugin is not going to be of much help here as it only pulls from one previous job. Perhaps one of the artifact publisher plugins might be useful, or you may need to add some custom build steps to put/get artifacts.
The group that I work in has standardized on Jenkins for Continuous Integration builds. Code check-in triggers a standard build, Cobertura analysis and publish to an Artifactory SNAPSHOT repo. I've just finished adding a new target to the master build file that'll kick off a Sonar run but I don't want that running on every check-in.
Is there a way to schedule a nightly build of a specific build target in Jenkins? Jenkins obviously facilitates scheduled builds but it'll run the project's regular build every time. I'd like to be able to schedule the Sonar build target to run nightly.
I could, of course, create a separate Jenkins project just to run the Sonar target on a schedule but I'm trying to avoid that if I can. Our Jenkins server already has several hundred builds on it; doubling that for the sake of scheduling nightly builds isn't very desirable. I looked for a Jenkins plug-in that might facilitate this but I couldn't find anything. Any suggestions?
Here's one way to do it, if you are ok with triggering the build using cron or some other scheduling tool:
Make the build parameterized, and use a parameter in your build file to decide if the Sonar build target should run or not.
Trigger the build remotely by HTTP POST:ing the parameter values as a form to http://[jenkins-host]/jobs/[jobname]/buildWithParameters. Depending on your Jenkins version and configuration, you might need to add an Authentication Token and include this in your url.
Authenticate your POST using a username and password.
wget --auth-no-challenge --http-user=USERNAME --http-password=PASSWORD "https://[jenkins-host]/job/[jobname]/buildWithParameters?token=<token defined in job configuration>&<param>=<value>&<param2>=<value2>"
I am also looking for a solution for this. My current solution in my mind is to create 2 triggers in the regular build, one is the nightly build, another one is Polling SCM
In the sonar plugin configuration, it has the options to skip the builds triggered by the SCM change. Therefore, only the nightly build will start a sonar analysis.
I didn't get a chance to test it now, but I suppose this will work.
Updated on 12/19/2011
The above solution doesn't work if the sonar analysis is invoked as a standalone build step. To make the sonar analysis run conditionally, you could use the following 2 plugins:
Conditional BuildStep Plugin - this allows the sonar analysis to be run conditionally
Jenkins Environment Injector Plug-in - this allows you to inject the variables to indicate how the build is triggered.
I am referred to Hudson today.
I have heard about continuous integration before, but I have no idea what the heck is a ci-server.
Hudson is really easy to install in Ubuntu and in several minutes I managed to set up an instance of it.
But I don't quite understand the workflow of a ci-server, or how am I supposed to use it?
Please tell me if you have experience about ci, thanks in advance.
Edit:
I am currently using Mercurial as my SCM, and I wonder what is the right way to use it with Hudson.
I have installed the Mercurial Plugin of Hudson, and I create a new job with a local repository. When I commit in the repository the Hudson job is built with the latest version of my source code.
If what I used is a remote repository, what's the workflow like?
Is it something like the following?
Set up a Hudson job with the repository
Developer makes a local clone of the repository
Developer commit and push changes
The remote repository update with the incoming changeset
Run a Hudson build
There may be something I misunderstanded at all, please help me point it out.
Continuous Integration is the process of "integrating software" continuously i.e. as frequently as possible (ultimately after each set of changes) to avoid any big-bang integration and all subsequent problems by getting immediate feedback.
To implement Continuous Integration, you first need to automate the build of your software (where build means of course compiling sources, packaging them, but also compiling tests, running the tests, running quality checks, etc, anything that will help to get feedback on the health of your code). Then you need to trigger the build on the latest version of the sources on a particular event (a change in the repository, a temporal event), to generate reports and to send notifications upon failure (by mail, twitter, etc).
And this is precisely the responsibility of a CI engine: offering trigger mechanisms, being able to get the latest version of the sources, running the build, generating and publishing reports, sending notifications. CI engines do implement this.
And because running a build is CPU and Disk intensive, CI engines usually run on a dedicated machine (or even a farm of machines if you want to build lots of projects).
Back to your question now. Once you've got Hudson running, configure it (Manage Hudson > Configure System): setup the JDK, build tools, etc. Then setup an Hudson Job and follow the steps: configure the location of the source repository, the build tool, the trigger, a notification channel and you're done (you can do more complex things but that's a start).
For more details on the setup, check:
The official Use Hudson guide for more details. << START HERE
Continuous Integration with Hudson - Tutorial.
Spot defects early with Continuous Integration.
Martin Fowler's overview of continuous integration is one of the canonical references. In my opinion, using automation to make sure your code base is healthy is one of the most useful things that you can set up.
Update Sorry that I didn't have much time earlier to expand on my reply. #Pascal_Thivent is right that in order to effectively use CI, you need to be able to automate your builds, tests, etc. CI is actually a good forcing function for this. For me, it's one of those little warning flags if I start to think that it would be too painful to put a build into Hudson. It means that something is not quite right.
What I like about Hudson is that it's flexible enough to accommodate different workflows. We use it for both builds / unit tests and releases. And it eliminates a lot of the worry about certain release procedures only working in one person's environment.
What I don't like about Hudson is that it is occasionally unstable when new builds break plugins. I've had a couple of upgrades (2 out of 10 or so) go bad because of incompatibilities. I do two things now:
I never upgrade my team's Hudson server to the latest and greatest right away. I generally only upgrade when there are significant new features, or bug fixes.
I now have a basic Hudson instance set up with all my plugins on a virtual machine with some dummy builds that I fire up to test out any new upgrades before doing it on the public server.
Is there a way to configure Hudson to only execute Build or Post Build actions if there are changes in SVN/CVS
Thank you
You can have Hudson poll the SCM for changes and only do things if it finds changes.
Poll SCM: Configure Hudson to poll changes in
SCM.
Note that this is going to be an
expensive operation for CVS, as every
polling requires Hudson to scan the
entire workspace and verify it with
the server. Consider setting up a
"push" trigger to avoid this overhead,
as described in this document
You can also add something to your SCM post-commit hooks that will fire off a Hudson build.
Trigger builds remotely (e.g., from
scripts):
Enable this option if you would like
to trigger new builds by accessing a
special predefined URL (convenient for
scripts).
One typical example for this feature
would be to trigger new build from the
source control system's hook script,
when somebody has just committed a
change into the repository, or from a
script that parses your source control
email notifications.
You'll need to provide an
authorization token in the form of a
string so that only those who know it
would be able to remotely trigger this
project's builds.
It is not as simple as looking at the revision number (as stated elsewhere) unless your build is for the entire subversion repository. Typically you have projects sharing a single subversion repository and you are building some sub-tree. The global revision number doesn't help.
'svn info [url_to_subtree]' will show the Last Changed Date. You can parse this and figure out if it is later than your last build date and trigger a new build.