I use the maven release plugin within CI/CD pipeline based on Gitlab.
Everything works fine but I try to inject some sensitive data directly to a file using runner during the pipeline run:
simply echo $var > file
yet before running maven plugin - it works fine but running maven release perform in one of its steps is branch checkout which means that all uncommitted files will be skipped. Of course, as they are sensitive ones I don’t wanna keep them directly at the repo level and finally as a result I have an artifact without necessary data
Maybe someone knows how to inject/add such a file running maven release plugin? Worth to add that I'm fully aware of fact that it's not a secure way to keep sensitive data in the artifact - it's only a temporary solution.
Related
I would like repeatable results when running maven commands locally, even if somebody else is pushing updates to a snapshot dependency.
To achieve this, I would like to use the updatePolicy of never.
This will allow any dependencies that aren't available locally to be downloaded, while any I have installed locally will be used.
The offline flag won't work in this situation, as there may be dependencies that I haven't installed locally which will need to be downloaded from the remote repo.
I don't want to have to modify the pom, as doing this locally with every checkout will be error prone, and I don't want to commit these changes as it will have adverse effects on other developers.
Ideally I'd like to specify this from the command line. The opposite of the -U flag.
I've searched the docs, and so far have not found out how to do this.
If you want repeatable builds you can create a Docker image that can run Maven. Then load all you project files and run Maven build.
This will provide a clean environment for your build every time.
About the changing dependencies, if you work using SNAPSHOT dependencies, you must expect this different results. That is what SNAPSHOT means: "this is under development".
If you (or your team) control the SNAPSHOT dependency and there is an error in the build that's a "good" sign, the tests found something to be fixed.
If you (or your team) don't control the SNAPSHOT dependency, you would prefer to the last stable release.
I have Jenkins version 2.7.1 running on a Windows 7 machine. It is successfully pulling code from a subversion repository and running tests. I have the test jobs set up for the development branch of each project only.
We periodically make stable releases of the projects in jar files with version numbers. I would like to have Jenkins be the repository manager for those stable releases. These are made by hand - There is no Jenkins job making or testing stable releases. The projects do use Maven.
Each stable build is tagged in the subversion repository, so it could be made again on demand if needed.
I downloaded the Maven repository server hoping to make this fit the purpose. I read the documentation that's provided, but it's pretty terse. As I understand it and have it configured now, this appears to have a couple of issues:
If I go to jenkins-ip/plugin/repository/project, it has made directories there that expose the names of all of my projects, which seems undesirable. (Here jenkins-ip is the IP where I access Jenkins on my local network.)
On the other hand, there's nothing but empty directories under these projects, so they're currently useless.
These projects all correspond to the continuous testing of the development branch. There's no apparent way to get the stable builds into the hierarchy. (It doesn't seem efficient to create a job for each stable release...)
Is there anyway to get Jenkins (with this plugin or through another method) to be the repository manager just for the stable builds? I know that I can start a different repository manager like archiva, but it would be ideal to use Jenkins since it's already running and it seems to claim capability for this function now.
To use Maven repository server you have to build the project on Jenkins.
Then the plugin will expose all archived artifacts as maven repo.
Note you need to use a "Maven project" type for it to work (freestyle is not supported)
There are several plugins that will help you manage building from multiple tags, however not all of them work with "Maven project" type.
You could also try Jenkins pipeline (previously "Workflow") or the Job-DSL plugin.
A simplest solution would be to have a build parameter specify the tag name (then checkout e.g. ^/tags/projectname/${tagParam}), but you have to figure out how to trigger the job then.
I have a Jenkins job that builds a simple Maven project. If all I do is build, it works just fine. The problem arises when I try and do a release, dry run or regular. It consistently fails with the Cannot prepare the release because you have local modifications error. I have wiped out the workspace, but the problem persists. Is there any way I can get Maven to tell me which file it thinks has been modified? I would assume that by wiping out the local workspace and immediately running the dry run release that there wouldn't be any opportunity for anything to get modified.
Please note, I do not have access to the Jenkins server or the slave that is running the actual release build, so I can't use any tools there (like SVN) to determine what is supposedly modified.
You can use the Maven SCM plugin to do a diff.
https://maven.apache.org/scm/maven-scm-plugin/diff-mojo.html
Basically, integrate the maven plugin upstream of the failure, and see if anything has been changed. I imagine you might be able to see the output in the log, but if you cannot, you might be able to move your "real" maven pom.xml aside and replace it with one that generates a diff file and with the help of the maven build helper plugin, attaches that file as an additional aritfact (to a pom target).
It turned out the solution to my problem was to not use the "Local to the workspace" strategy for my private Maven repository in the Jenkins job configuration. By changing that to the "Local to the executor" strategy the problem went away. I'm still not sure why it was having the problem in the workspace, but this solution resolved it form me, and might work for others.
Wondering how people manage their project artefacts through an environment lifecycle of say DEV - AQA - CQA - RELEASE and if there's some best practices to follow.
I use a Jenkins build server to build my projects (code checkout then maven build). My artefacts all have version 1.0.0-SNAPSHOT and are published to a local .m2 repo on the build server. There are also Jenkins jobs that rebuild the DEV system (on the same server) using those artefacts. The project build is automated whenever someone checks in code. The DEV build is automated on a nightly basis.
At some point, my lead developer determines that our project is fit to go to AQA (the first level of testing environment on a different server).
For this I need to mark the artefacts as version 1.0.0-1 and publish to a remote AQA repository (it's actually a Nexus repo).
The Maven deploy plugin sounds like the right approach, but how do I change the version number to be effectively 1.0.0-$release (where $release is just an incrementing number starting from 1)? Would Maven/Nexus be able to manage the value of $release, or would I need a simple properties file in my project to store/update the last used $release.
Furthermore, someone tests AQA and determines its fit to move on to CQA (second testing env). This is 'promote to AQA'. So my requirement is to copy the artefact from the AQA Nexus repo and publish to the CQA Nexus repo.
Likewise, after CQA, there'd be a 'promote to RELEASE' job too.
I think the version value remains unchanged during the 'promote' phases. I'd expect the AQA repo to see all versions 1-50, but CQA only 25 and 50, then RELEASE only 50, for example.
I can find loads of info about Maven plugins/goals/phases, but very little about a prescriptive method on how or where to use outside of the immediate development environment.
Any suggestions gratefully received.
Staging/promoting is out of scope for Maven. Once deployed/uploaded to a remote repository, that system is responsible for the completion of the release cycle. Read this chapter about staging: http://books.sonatype.com/nexus-book/reference/staging.html if you use Nexus.
Build numbers are just that build numbers. They are not promotion / staging numbers.
You should come up with another means of tracking your promotions, because otherwise one might get confused in "knowing" that build 10.1.4-2 is the same as 10.1.4-6. Certainly, all of the Maven related software will see those two builds as different builds.
In addition, if a person "grabs" the wrong copy of the build, the way you are managing staging within your build number will increase confusion. As if you don't kill all of the 10.1.4-2 builds, then someone might get a copy of that not realizing that the build has been promoted to 10.1.4-6. This means that for the "last" staging number to be the most likely one to be grabbed, you must do two things (which are impossible in combination)
Remove all the old staging numbers, updating them to the new ones.
Ensure that no copy of an old staging number escaped the update.
Since people generally can copy files without being tracked, or said files might not be reachable at time up "update", or timing between reaching all the files cannot be simultaneous, such a system is doomed to fail.
Instead, I recommend (if you must track by file), placing the same file in different "staging directories". This defines release gateways by whether the file exists in a certain directory, and makes it clear that it is the same file that is going through the entire process. In addition, it becomes easy to have various stages of verification poll their respective directories (and you can write Jenkins tasks to promote from one directory to another, if you really wish).
We currently use SilkCentral Test Manager (SCTM) integrated with our source control system via SCTM source control profiles. However, we would like to explore integrating with build artifacts checked into Maven's remote Nexus repository instead.
The idea being that the application-under-test is built and checked into Nexus along with the automated tests only if the build and the tests pass. Therefore, when QA is ready to run tests from SCTM (manual or automated), there is a well-defined combination of application build artifacts and test build artifacts in Nexus that present a more reliable target for SCTM as compared to getting the latest code from the source control system.
All of this is more relevant during active development when the code and the tests and changing daily and the builds are snapshot builds rather than formal builds with tags in the source control system that SCTM could use.
SCTM apparently has support for both universal naming convention (UNC) and Apache virtual file system (VFS) and either of these should potentially be utilizable to point the SCTM source control profiles to Nexus artifacts rather than raw source code. However, I wanted to check with the community to see if there's a simpler approach. (For example, I noted the existence of a Hudson SCTM plugin.) Also, I welcome alternative thoughts and ideas.
There are probably many solutions for solving this, I'd try the following:
Manage the build/first test/publishing steps in Hudson/Jenkins.
For example by modelling it with dependent jobs, the publish job is only triggered if the tests pass. There are also more advanced gatekeeper plugins available (for example a Downstream Ext plugin) which might solve this even more comfortable.
Once the publishing is done, use the Hudson/Jenkins-Silk Central plugin to trigger the executions on Silk Central. There, instead of using UNC or VFS, I'd rather use a setup script which pulls the artifacts from the repository and prepares everything for the tests. This would allow you to use something Maven/Nexus aware to pull the correct artifacts from the repository, instead of somehow trying to make it accessible via UNC or VFS.