Running same jenkins job with different source code but in parellel - maven

I have a jenkins job that do build and deploy the maven project. The job depends on few parameters that I pass to the job. This job also checks out the code from git repository.
I have 4 branches ( dev, test, release and patch) as the source code of the project.
How can I run the same job with different parameters and different source code of the project.
Example :
- let say we triggered the job to run by passing param1 and param2 to the job and using the dev branch.
taking into consideration that the run from step 1 hasn't completed , how can I trigger the same job by passing param3 and param4 but using test branch this time
I want to do different build from different source branches from the same job in parallel .
Any other suggested design ?

One inherent problem here is that - when you fire another build of the same job, it will overwrite the workspace it is using.
There are two ways of tackling the issue IMO:
Like #bish mentioned in his comment - two separate jobs.
Pass a parameter that will kick off a small script that will checkout the project to a workspace/subdir folder instead of workspace. You can do this in the Add pre-build step -> Execute shell script. Here, you need to ensure that you don't 'wipeout the workspace folder'.
(The Jenkins GIT plugin has an option to checkout the project to a subdirectory, but I couldn't figure out how to trigger that conditionally. So, going by a simple manual intervention here.)
Next, you can choose to give a list of the branches available as shown in one of my earlier answers
Let me know if this helps.

Related

Is there a way to specify several Jenkins files for the pipeline?

At the moment, we are generating Jenkins jobs using the Job DSL plugin. Typically, we have the following jobs per project:
CI build (SNAPSHOT build)
Deployment, one per stage
Integration test (nightly build)
Creation of a release
Reports (Maven site, during the night)
Am I right that there can be only one Jenkins file in the project's repository? How could I map our requirements to the new Jenkins pipeline?
I'm asking because we're going to install version 2 of Jenkins, and I'm not sure whether we should abandon our Jenkins job generation and use Jenkins files.
There are a couple of options, which might help you to migrate over to Jenkins pipelines. But you don't have to, especially not all at once.
You can use a shared library to define functions that can be used in multiple jobs, e.g. a buildOurThing function.
You can trigger an existing job (of whatever kind) using the build step. So you could model your new pipeline around existing jobs.
You can still use parameterized builds, if you want to use the deployment job with different targets.
Jenkins pipeline is really worth using, but maybe don't force you to an immediate switch. If JobDSL works for you, keep it. If you have new jobs (pipelines) to create, get familiar with pipelines.

Is there a build step in TeamCity that exits the build successfully without performing additional build steps?

I have a project that needs to execute either three or four build steps depending on the branch in source control. More specifically, if I'm merging in a PR and running the build (for GitHub status notifications) I have one extra build step that is required.
It's that last build step that I need to omit if it's a non-PR branch.
Is there a way to add a build step that checks the trigger and exits the build successfully? Or a way to exclude a build step based on a branch filter?
You can check the condition and modify the step logic inside the build script. See the related ticket and an example of the script.
BTW, It is not a good practice to change the logic of the build inside build script. In this case you no longer "compare" builds in the build configuration: they start to form multiple unrelated sequences. Also the statistics of the builds will be uninformative. The recommended setup is to create several build configurations based on template.
Depending on which type of runner you are launching, but you can, in some cases, add few lines of code to get your current branch name with the property : %teamcity.build.branch%
In my case, I just add this as an extra parameter for powershell scripts and if this is a number, do something, else, do other stuff. ;)

Jenkins different schedule for different configurations

I am working with a multi-configuration Jenkins project. The two configuration axes are Win/Linux and 32/64-bit. I would like to build the primary configuration (32-bit Windows) whenever version control changes, but to only build the other configurations once weekly (just to make sure that they stay reasonably up to date).
Is it possible to achieve this schedule without breaking the project up into multiple individual projects?
Unfortunately not through Jenkins directly at this point AFAIK. There is only one time handler per job, and the multi-configuration is a single job, it has a single timer.
There is a hack, but it will be tough, and I'm not sure of the exact script required, but if you can check the day of the week, you could try something like this in your script:
if (day == Sunday |OR| $NODE_NAME == win32), then:
<carry out build steps here>
finish
This way:
If the day is Sunday, all steps will be carried out regardless of Node.
If the node name is win32, the step swill be carried out too.
However, if day is not Sunday and $NODE_NAME is win64 or linux32 or linux64, no steps will be carried out.
Note that $NODE_NAME is a standard Jenkins environment variable. However, this assumes that your build is done through either "Execute Shell" or "Execute Windows Batch"
Is there any reason the others shouldn't build at the same time?
You could create two jobs, one for your primary so it is a free-style job, and the other as a multi-configuration with Win64/Linux and leave that on a separate, weekly timer.
Please consider this:
Why not build ALL available configurations EVERY time?
- after all, that is the whole idea of Continuous Integration...
Can drop the artifacts of those builds after a short period of time,
so they don't clog your disk, but if anything breaks your build - you will know it right away.
Can also set the slaves' queue to run a single job at a time, so the builds don't overload the build-servers.
The other solution requires:
Two "starters" for your build
All slaves to be available
Set Job_A1 as a weekly scheduler that triggers Job_B (the main multi-configuration build process).
Set Job_A2 to run Job_B whenever there are source-code changes.
Set Job_B to know if it was triggered by Job_A1 or Job_A2 (can pass its name as a parameter),
and also set Job_B to ignore calls from Job_A2 ("exit 0"), if the current config. differs from Windows-32bit.
This way, all 4 configurations will run whenever there are source-control changes, but only one will actually build.
Good luck!

How to split a big Jenkins job/project into smaller jobs without compromising functuality?

we're trying to improve our Jenkins setup. So far we have two directories: /plugins and /tests.
Our project is a multi-module project of Eclipse Plugins. The test plugins in the /tests folder are fragment projects dependent on their corresponding productive code plugins in /plugins.
Until now, we had just one Jenkins job which checked out both /plugins and /tests, built all of them and produced the Surefire results etc.
We're now thinking about splitting the project into smaller jobs corresponding to features we provide. It seems that the way we tried to do it is suboptimal.
We tried the following:
We created a job for the core feature. This job checks out the whole /plugins and /tests directories and only builds the plugins the feature is comprised of. This job has a separate pom.xml which defines the core artifact and tells about the modules contained in the feature.
We created a separate job for the tests that should be run on the feature plugins. This job uses the cloned workspace from the core job. This job is to be run after the core feature is built.
I somehow think this is less than optimal.
For instance, only the core job can update the checked out files. If only the tests are updated, the core feature does not need to be built again, but it will be.
As soon as I have a feature which is dependent on the core feature, this feature would either need to use a clone of the core feature workspace or check out its own copy of /plugins and /tests, which would lead to bloat.
Using a cloned workspace, I can't update my sources. So when I have a feature depending on another feature, I can only do the job if the core feature is updated and built.
I think I'm missing some basic stuff here. Can someone help? There definitely is an easier way for this.
EDIT: I'll try to formulate what I think would ideally happen if everything works:
check if the feature components have changed (i.e. an update on them is possible)
if changed, build the feature
Build the dependent features, if necessary (i.e. check ob corresponding job)
Build the feature itself
if build successful, start feature test job
let me see the results of the test job in the feature job
Finally, the project job should
do a nightly build
check out all sources from /plugins and /tests
build all, test all, send results to Sonar
Additionally, it would be neat if the nightly build was unnecessary because the builds and test results of the projects' features would be combined in the project job results.
Is something like this possible?
Starting from the end of the question. I would keep a separate nightly job that does a clean check-out (gets rid of any generated stuff before check-out), builds everything from scratch, and runs all tests. If you aren't doing a clean build, you can't guarantee that what is checked into your repository really builds.
check if the feature components have changed (i.e. an update on them is possible)
if changed, build the feature
Build the dependent features, if necessary (i.e. check ob corresponding job)
Build the feature itself
if build successful, start feature test job
let me see the results of the test job in the feature job
[I am assuming that by "dependent features" in 1 you mean the things needed by the "feature" in 2.]
To do this, I would say that you have multiple jobs.
a job for every individual feature and every dependent feature that simply builds that feature. The jobs should be started by SCM changes for the (dependent) feature.
I wouldn't keep separate test jobs from compile jobs. It allows the possibility that successfully compiled code is never tested. Instead, I would rely on the fact that wen a build step fails in Jenkins, it normally aborts further build steps.
The trick is going to be in how you thread all of these together.
Let's say we have a feature and it's build job called F1 that is built on 2 dependent features DF1.1 and DF1.2, each with their own build jobs.
Both DF1.1 and DF1.2 should be configured to trigger the build of F1.
F1 should be configured to get the artifacts it needs from the latest successful DF1.1 and DF1.2 builds. Unfortunately, the very nice "Clone SCM" plugin is not going to be of much help here as it only pulls from one previous job. Perhaps one of the artifact publisher plugins might be useful, or you may need to add some custom build steps to put/get artifacts.

How to add some prebuild steps to jenkins?

I am a Jenkins newbie and need a little hand holding because we only maintain parts of our app in SVN. I have basic Jenkins install setup.
This is what I do to get a local DEV environment setup and need that translated to Jenkins in order to make a build:
DO SVN checkout (and get the 2 folders that are under SVN)
Delete the folders
Copy over the full app from FTP location
Do SVN restore
download sql file
Import into MySQL
How would I get the above mentioned steps in Jenkins? I know there are some post build steps that I can use. Just not sure how to put it all together. Any help will be much appreciated.
Tell Jenkins about the SVN repository and it will check it out automatically when a new build is started. That should take care of 1. 2-5 would be build steps (i.e. execute shell commands). Basically, you can set up Jenkins to do exactly what you do on the command line, except that the first step is taken care of automatically if you tell Jenkins about the repository.
Rather than trying to do these sort of things in Jenkins, you'll likely save yourself some trouble if you make use of something like Ant or NAnt to handle the complexities for your build.
I've found that doing my builds this way gives me added flexibility (ie, if it can be done via the command-line, I can use it in my build, rather than needing a Jenkins plugin to support it), and makes maintenance easier as well (since my NAnt scripts become part of the project and are checked into the VCS system, I can go back if I make a change that doesn't work out.
Jenkins has some build-history plugins, but over time I've found it easier to keep the majority of my 'build' logic and complexity outside of the CI environment and just call into it instead.

Resources