In Jenkins (or Hudson), I set up a pipeline of parameterized jobs, say A -> B, that copy artifacts from each other using the Copy Artifact Plugin. All jobs use the same parameter, e.g. the target platform. Each job is set to discard old builds and artifacts to limit disk usage, keeping only the latest stable build artifacts. I use parameterized builds to avoid having to maintain many jobs that only differ in the parameter setting.
This setup only works if all builds run after each other in a pipeline with the same parameter settings, and hence the latest build artifacts match the platform parameter setting of a dependent job. Now if someone first builds job A for platform x, and then for platform y, then builds B for platform x, the artifacts from A for platform x are already discarded and hence B cannot copy these.
Is there a way to tell Jenkins to keep the latest artifacts for each build with different parameters and discard all others?
Manual solution: copy the artifacts from A into a directory that is keyed by the parameters. Job A will clean out the directory before the build - and thus make sure that stale artifacts do not pass to job B. Job B will clean out the directory after it runs in any case.
Related
I am trying to setup a Jenkins pipeline to trigger builds using gradle for multiple environments.My requirement is that the artifacts produced when I run gradlew clean build should produce artifacts with name indicating the environment for which the pipeline was run. Example my-application-dev.jar
The value of the environment would be selected by the user when build will be triggered.
What is the optimal way to achieve this ? Does build allow to configure any such property via command line or do I need to define a task in my build.gradle and define properties within that task for which I will pass value from command line
There are basically two ways.
The first one is to pass these naming-relevant pieces of information to the gradlew process, e.g. via -D or -P properties.
Then you need the Gradle build to be aware of these parameters and craft the artifact names it produces accordingly.
The second one is arguably better and more contained. You simply rename the artifacts produced by the gradlew command after it completes, in the Jenkinsfile. This works well if the pipeline decides what to do with these artifacts (e.g. publish to a repository) as opposed to the gradle script doing it (in which case you would most likely be better off using the first method).
What is the set up required to move files between builds in TC? I am needing to move both modified source files and build binaries between the build configurations of a build chain.
I have 1 project with 4 builds. The builds are
Update Version Number (This build updates 15 sources files)
Compile (This build compiles a dozen objects)
Test (This build runs a regression test)
Create Package (This build creates a setup.exe file)
Information about the TC setup and chain
I am using perforce as my VCS.
All 4 builds use the same VCS root.
On all 4 builds under version control settings I have "Clean all files before build" set to "On".
"Update Version Number" build is triggered by any check in to the VCS. (This works)
I have been able to successfully chain and trigger the builds. However each build starts with a fresh copy of files from the VCS.
The chaining is set up to use snapshot dependency.
Based off of the TC documentation it looks like I should be using snapshot dependency and not artifact dependency. If I put the build steps of all the builds into the same build everything works. However we are looking to have more flexibly and expand on this build chain in the future.
I tried setting up the configuration so only the first build is attached to the VCS root and the other builds don't have any VCS root. This didn't work.
I have been unable to find the answer googling but I have been able to find someone else who is struggling with this problem. Sadly they didn't receive an answer.
After speaking with TC customer support I learned the correct technique is to use both artifact dependency with "build from the same chain" selected and snapshot dependency.
For example, project A generates two artifacts processor.exe and t.txt. Then in project B, can I add a build step to execute processor.exe t.txt?
I know there are two Runner types(.NET Process Runner and Command Line) that can execute programs. But how to get the paths of these artifacts?
Yes, I think it should be possible to run with Command Line runner *.exe file generated by other build.
You just have to make sure that build agent which runs Project A outputs these artifacts to place where build agent which runs Project B has access (in case if you have single agent it's not a concern obviously). And probably placing these artifacts into agent's working directory is not the best place because it can be cleared by doing clean checkout from VCS. Just choose some generic directory on the server and specify it for artifact output in Project A and then for Command Line runner in Project B.
Consider a Gradle plugin that adds three tasks to a project - a buildZip task to create a distributable zip of the project, a publishZip task to publish that zip to a shared repository, and a cleanZip task to clean up any local version of the zip. For local development, cleanZip buildZip will be used frequently, but the automated build system will be running buildZip publishZip cleanZip.
One of the projects in which this plugin is being used wants to run their build using Gradle's parallel flag to allow the different parts of the project to be built in parallel. Unfortunately, this runs into a problem with the zip tasks - buildZip depends on the project actually building, but cleanZip doesn't have any dependencies so it can run right away, leading to the automated build system not being able to clean up.
Declaring any dependencies between these tasks isn't a good idea because they should be able to be run separately. Also, I can't specify mustRunAfter (at least between buildZip and cleanZip) because sometimes clean should be first and sometimes build should be first.
How can I tell Gradle what order to run these tasks in, in a way that will be honored by --parallel and isn't hardcoded to have a particular one always run before the other?
What you can do is: detect if gradle is run with --parallel and based on this configure dependencies between tasks appropriately. It can be done in the following way:
println project.gradle.startParameter.parallelProjectExecutionEnabled
It might be the case here some of things I have in my mind and in this subject has already been discussed here, but I really would be very thankfull to have step-by-step answer on achieving the following efect on Jenkins CI, First of all - here is tools I can operate with:
Two Jenkins CI servers - Jenkins A and Jenkins B ;
Jenkins B cannot serve as data storage for builds, therefore the artifacts, test, etc. should be saved on Jenkins A :( ;
Apache Ant ( latest ) and NAnt ( latest ) .
The goal is to achieve the following: Ability to start build on Jenkins A, make source code and build on Jenkins B, but archive and show the artifacts again on Jenkins A.
As much I have googled, I found there is term widely used - "distributed build" ... is it the case here, too?
Currently using latest Jenkis CI version available on both machines ( Jenkins A and Jenkins B ).
What job config I need to use to be able to do such multi-Jenkins setup under my company domain?
You do not want to have a full Jenkins install on both machine.
What is sounds like to me is you should be installing Jenkins on machine A, and set up machine B as a slave node to that Jenkins installation. The Jenkins UI makes this very intuitive, but if you need details they are available here:
https://wiki.jenkins-ci.org/display/JENKINS/Step+by+step+guide+to+set+up+master+and+slave+machines
https://wiki.jenkins-ci.org/display/JENKINS/Distributed+builds
Once you've done that, you will need to explicitly configure your job so that it:
Archives whatever artifacts you need archived (you probably have this done already).
Builds only on the machine B slave node by entering the node name in "Restrict where this job can be built."