What I want to achieve is two step build automation
Step A - Build & Upload to artifact repo
Create build job in Jenkins which will create build after every check-in
Upload every successful build on Archiva server
Step B - Get latest artifact & deploy on required servers
After every desired interval, get latest build from Archiva
Deploy build uploaded in Step A.2 to dev/qa/stg server by unzipping its content in web server directory.
I was able to do achieve Step A by using maven goals in my project pom.xml but any idea/suggestions/best practices for Step B.
I understand/agree, I would need two different jobs having different pom.xml, question remains how will we get latest war from repo in pom and how will be deploy that latest war on remote server by unzipping it, as tomcat there does not have admin module.
I would deploy not a jar/war artifact on step 2, but would create a RPM that contains a needed files structure.
With maven it's quite easy to do with a maven rpm plugin
3-4. Nexus has built in YUM repository support, so you can use yum to install the latest rpm version
So I've always found it better to separate the builds and the deploys.
The schedule for those can be independently managed then.
Assuming you are using linux on the servers you could use the ssh-plugins in jenkins to download the artifact in archiva
wget http://server/repository/internal/group/artifact/version/artifact-version.jar
As for the deploy, you could sftp them over to the deploy server also using jenkins SFTP plugin.
Related
I usually use mvn versions:use-latest-versions command to update my dependencies to the latest ones which other teams have been deployed to our free Jfrog's Artifactory server on our local address : http://192.168.100.243:8082/artifactory/X_Douran/.
My problem is when I deploy an artifact (a jar file) with Artifactory UI or with curl (using Jfrog's Rest Api), the command mvn versions:use-latest-versions doesn't work correctly and do not update my pom but when I run mvn clean deploy on my source code of my dependent project then running mvn versions:use-latest-versions on my final project it works correctly and do update my dependency in my pom.
So I want to know what is the different between deploying via Artifactory UI and deploying via mvn clean deploy ?
You always need to deploy the POM along the JAR, otherwise Maven will not work correctly with these dependencies. Furthermore, you need to make sure that the metadata files are updated. I am not sure that Artifactory does this if you deploy using curl or the UI.
Deploying your own JARs regularly through the UI is not recommended. You should build them on a build server (like Jenkins) and then automatically deploy them to Artifactory.
Changing JAR files "by hand" should be forbidden.
I need to build a Java project on Maven. I am working on a multi-module Maven project that's built on the Jenkins Pipeline in the Nexus repository. I have a few libraries that are not available on the Nexus repository. I can't manually upload the libraries. I am building this project on a pipeline.
What I did:
I created a folder named jars in the project root of the Git hub repository and manually put all the jar files that are not available on Nexus. In the dependency, I referenced all these local jars as in the dependency parameters.
In the repositories, I gave the URL of the git hub repo of the jar folder. The Jenkins were not able to pick the libraries. I am getting the following error: dependency: dependency version - Build Error - Could not build for non-released dependencies and I am getting an error for all the jars that are in the jars folder. I tried putting the jars folder in src/main/resources but still getting the same error.
How can I reference this jar folder so that the Jenkins Pipeline can take it? I don't have control over the Jenkins / Scripts that are involved. I am a developer just building it on the Pipeline.
P.S: I don't have access to the internet at my company to post the POM or Build Failure errors.
Adding more details:
It's built on the Pipeline. There are two repositories: Nexus 2 and Nexus 3. The particular libraries are not available on Nexus 3 and pipeline takes the build only on Nexus 3.
We have raised a request to upload those libraries but it's not going to happen anytime soon. The Jenkins Pipelines takes it's files from the Github repository and builds the Java project using Maven. I don't have control to a pipeline or any of the scripts in Jenkins.
We downloaded all the libraries that are not available and put that in a folder in git hub. There are 4 cycles in the Pipeline. Github Cycle / Jenkins Cycle / Deployment Cycle / Release Cycle.
Github Cycle: In this cycle, it follows three stages. It takes the code from the code, builds it. It builds the snapshot and uploads it to Nexus repo. In these 2 stages, it was able to successfully build by taking the code from the GitHub and builds it and artifact generated. Third stage: It's really strange as in this stage, it again builds and build getting failed in this stage citing code for Non Released Dependencies for the jars that are uploaded in the git hub.
What might be the reason for this: When it can build in the first two stages of the Github cycle and getting failed in the third stage for Build Failure for non-released dependencies.
The pipeline is designed in such a way that it looks only on Nexus 3 and build during each phase of the cycle.
In the repositories, I gave the URL of the git hub repo of the jar
folder
That does not work because your lib folder is not a valid Maven repository.
How can I reference this jar folder so that the Jenkins Pipeline can
take it?
You have some options:
Set up custom Maven repository manager. You can use Nexus
Repository Manager or JFrog Artifactory or something else.
It will give you the greatest flexibility and allow to do a lot more
in the future. Downside is, you will need to have the infrastructure
to run this which usually comes with some sort of maintenance cost.
Install the bundles in Maven's local repo from the jar folder you already have. There are two ways you can do that:
Via script in Jenkins Pipeline that runs before your build and calls mvn install-file ... for each library in your jar folder. You can find the exact syntax for this command on Apache Maven Install Plugin site
By changing your build and calling the install-file goal of the maven-install-plugin in earlier build phase. I've personally
never done that but this answer suggests it's possible.
remove the files from the jar folder and create a wrapper project for each of them which does nothing but install the jar in
the local maven repository. Make sure those are the first modules to
run in your multi-module project.
We are using Jenkins to build (maven) & deploy artifacts (JARs & *WAR*s) to an in-house artifactory server (both snapshots and releases).
For deployment, currently, we got Jenkins jobs that package the war file (from a release scm tag) and deploy to different environments/servers. We want to skip the package phase as it seems unnecessary to package it again & again for a released version because it's not possible to get a different copy of war file even after trying 1000 times.
We are looking for a way in Jenkins to get the artifact (war) from Artifactory and deploy it to a container. I am sure other people would have faced this situation too but I am not able to find any online material regarding this.
Is there any Jenkins plugin that takes a war file from Artifactory (based on a version) and deploy it to a remote container?
If this is not the right way of doing it then what are the recommendations for any other approach?
Thanks
I don't know about a plugin which takes a version # and deploys that, but you can build a Jenkins job to deploy the last successful release to a previous environment (thus copying from DEV-->QA for example.)
To do this, you would use the copy-artifact-plugin.
Here's an easy to follow run-through of this kind of setup:
http://www.lordofthejars.com/2012/09/deploying-jee-artifacts-with-jenkins.html
Every artifact stored in Artifactory will have a unique URL that includes the version number. It will take the format
http://artifactory-server/repository-name/path-to-artifact/version/filename
e.g.
http://artifactory/apps-releases-local/com/yourorg/yourapp/1.5.67/webapp.war
(depending on how you do your packaging, the WAR file name may include the version number as well).
So your deployment job can construct the Artifactory URL and download the file. Depending on how you have security set up in Artifactory, you may need to authenticate the request.
I am in a situation where I have one Build Server box which is to carry out all continuous integration and manage our maven repository. The box works as follows:
There is one maven repository which is hosted through Apache Server as a URL for developers to use
All Jenkins jobs (including release jobs) run mvn install so that artifacts are kept in this one repository.
I would like to get rid of the Apache server and run Nexus on this same box to manage and host repositories, however I have the following questions/ideas:
With Nexus and Jenkins on the same box, will it mean that I will have to manage two repositories, one where maven installs an artifact to a local repository, and one where maven deploys an artifact to nexus? Would it be possible to have Nexus manage the "mvn install" repository also? How can I make sure we don't run out of disk space on the server very very quickly all the time?
Thanks
Added as response to comments: Thank you both, I am thinking I will just set the Jenkins jobs and release plugin goals to mvn package deploy:deploy in order to skip the install phase, that way, artifacts go directly from the target directory to Nexus. However I guess the Jenkins job will require a local repository from which to use depedencies which will get copied from Nexus to the maven local repository during the build, I am not sure if this can be avoided though.
mvn install installs in the local repository
mvn deploy installs to the remote repository
these semantics are defined in the lifecycle and map to different plugins. Their implementations are different.
You don't have to manage the local repository. Actually for some if not most jobs you might even want to define it localized to the job (with the 'Use private Maven repository' option) instead of to the user who is running the job, especially that you plan to use nexus for repository.
You will have to change your jobs to use mvn deploy instead.
How can I make sure we don't run out of disk space on the server very
very quickly all the time?
Configure Jenkins/Nexus. Discard old builds and disable automatic artifact archiving. Both settings can be found in the Jenkins job-configuration. Also you could delete old artifacts automatically from Nexus using Scheduling Tasks.
There is no need to install the artifacts into the local maven repository when using Jenkins/Nexus on a dedicated server.
I'm trying to setup a complete CI server, but I struggle on some points.
Currently, my system work as follow :
I commit local changes on my local GIT repository, then push to the GIT repository on the CI server
I then have a jenkins job triggered by the SCM change, who run a clean install and by doing so executes all my Junit and Jstestdriver test (via a local jstd server). This job deploy the snapshot artifact to a repository on my nexus repository
I installed M2-release-plugin for jenkins, and setup my pom.xml accordingly using maven-release-plugin. When i click on "Perform maven release" in jenkins job page, jenkins run mvn release:prepare release:perform, thus creating a tag in my git repo (say v000001) and deploying a versionned artifact on my nexus repository.
I don't really know if this process is fine, but i guess so...
My problem is that I want to deliver the versionned artifact in my nexus repo (say "artifact-v0000001.war") in my production tomcat. But I can't figure out how to do it.
When I do "mvn release:prepare release:perform tomcat:deploy" it deploys the new SNAPSHOT artifact built ... I don't want to do this, I want to reuse the artifact from the nexus repository.
Is there a way to doing this using a tool (maven/jenkins plugin, or external)?
Basically, I want to fetch the last release artifact on the repository, and send it to the tomcat manager for dereploying the webapp.
Do I need to setup a delivery job separated from the release job?
Jenkins, especially when combined with a tool like ANT, can do just about anything. It has a lot of plug-ins, and you can always write a script and incorporate it into a Jenkins build. Currently, I use Jenkins to deploy web applications to Windows IIS servers. What you could do here is have a Jenkins build that has your SVN path set in the source control section so that it fetches the latest version when you trigger the build. From there it should be fairly trivial to write an ANT script that copies it over the existing JAR in Tomcat, which will automatically restart it.
Your problem is that you are probably using the Jenkins release plugin, not the "m2 release plugin". The problem with the standard plugin is that it performs the regular build, saves the artifacts, then performs the release. It will then try to deploy the wrong artifacts that it created from the regular build.
The m2 release plugin solves this particular problem. There are some tricky workaround for this problem, but that's how it stands at the moment until this feature is implemented: https://issues.jenkins-ci.org/browse/JENKINS-11120 (log in and vote for it!)