We are planning to implement continuous integration with Jenkins software, so we successfully installed Jenkins in our server (windows platform)
My question was
How can I manage (keep) my source code?
How can we configure if the source code located in some other machine / server / cloud
Our current process
Jenkins, source code is available in same machine, so we created a project in Jenkins and map the pom.xml (like D:\pom.xml) file under build section.
Choose a Version control tool, consider that SVN and GIT are quite different (Git--> Distributed system, SVN-> Centralized system) so read a bit about them before choose, then check for the availability of GIT/SVN plugin in Jenkins unless it already contained in your Jenkins installation.
Put the code under SVN/GIT, follow the standards, configure Jenkins in order to access to SVN/GIT server.
Check how to configure maven with Jenkins, it has a great integration with Maven.
Maven Jenkins:
https://www.tutorialspoint.com/jenkins/jenkins_maven_setup.htm
Jenkins Git:
Jenkins and Git sparse checkouts
The flow of the single build job is:
Checkout The code,
Specify the folder with the pom.xml file
Run the maven goal you need (e.g. mvn install)
You put your source code into Subversion or Git. Then you give your Jenkins access to the Subversion/Git and point it to the URL which you want to checkout.
Answer to first question : Host your code in a SVN or GIT repository. Git has higher market share than SVN because of large feature set and higher efficiency. SVN is simple and easy to use for new comers in the version control territory. Explore the options and host the corresponding server in your infrastructure.
Answer to second question : There are multiple plugins available in Jenkins to fetch code from remote repositories. For git , there is git client, gitlab and gitHub plugin . For SVN there is Polarion webClient for SVN plugin.
Related
Is there a way to configure a single Jenkins job to perform the release of each newer version of my application?
i.e, I would like to know in detail if it's possible to do the following tasks without any human intervention. I'm using SVN as well as Artifactory.
Branch from the tag to be released
Change the snapshot versions in pom files to release versions (for each dependency defined in pom)
Take release build (EAR)
Deploy it in Weblogic instance.
Prepare release note
Tag the release
Thanks in advance.
It can be fully automated using Multijob plugin (Also with a regular single job, but using Multijob it will be easier and you can use Maven targets instead of some manual shell scripting ).
The workflow starts with:
commit and push to git
hook in git should trigger a job by http POST, so u need to configure your job to accept remote triggering. You can pass the branch name as parameter
job started, cloning the branch (git plugin)
start Maven project to mvn clean install and check unit tests (optional)
start Maven project to mvn release:prepare and mvn release:perform to omit the SNAPSHOT from pom.xml
tag the branch (using shell block and simple git commands)
merge to master branch (optionally)
start Maven project to mvn deploy to deploy to weblogic (weblogic should be configured in ~/.m2/settings.xml as the repository for deploying the artifacts
NOTE: all tasks can be also in 1 job with some shell scripting. the best practices is to use plugins but sometimes you will find it easier to use shell scripting for some tasks.
I'm getting started with Artifactory and am getting confused by all the terminology. What I'm trying to accomplish is the following:
$ git clone https://github.com/apache/hbase.git
$ cd hbase
$ git checkout branch-1.2
$ mvn clean install -DskipTests assembly:single
I'd like the git clone of this upstream project to live in Artifactory along with the components that Maven is pulling down. Additionally the final product zip file that mvn builds, hbase-assembly/target/hbase-1.1.6-bin.tar.gz, I want to store that in Artifactory too so that we can use it downstream in several CI jobs.
Questions
Is this approach correct?
How do I accomplish this?
Artifactory provides you with the Remote Repository functionality which you can also use to proxy various VCS services although I fail to see the point in just storing a clone of a git repo in Artifactory -
The VCS remotes are mainly used for cases where you directly include source code that you don't need to influence in your own code (i.e you can't git push changes back into Artifactory - they will not be persisted in the upstream git repo).
If you're aiming to have your CI job clone a certain branch, run tests, build it and then deploy build artifacts back to artifactory, I would suggest configuring your job to clone the git repo (using git not Artifactory) and then deploying the build artifacts back to Artifactory for downstream jobs to use.
Artifactory provides very extensive integration with popular build servers through which you can achive what's discussed here, your build artifacts should go into a Maven repository and the downstream jobs should reference it with their own pom files - you can quickly setup any Maven client to work with Artifactory using the Set Me Up dialog.
I am in a situation where I have one Build Server box which is to carry out all continuous integration and manage our maven repository. The box works as follows:
There is one maven repository which is hosted through Apache Server as a URL for developers to use
All Jenkins jobs (including release jobs) run mvn install so that artifacts are kept in this one repository.
I would like to get rid of the Apache server and run Nexus on this same box to manage and host repositories, however I have the following questions/ideas:
With Nexus and Jenkins on the same box, will it mean that I will have to manage two repositories, one where maven installs an artifact to a local repository, and one where maven deploys an artifact to nexus? Would it be possible to have Nexus manage the "mvn install" repository also? How can I make sure we don't run out of disk space on the server very very quickly all the time?
Thanks
Added as response to comments: Thank you both, I am thinking I will just set the Jenkins jobs and release plugin goals to mvn package deploy:deploy in order to skip the install phase, that way, artifacts go directly from the target directory to Nexus. However I guess the Jenkins job will require a local repository from which to use depedencies which will get copied from Nexus to the maven local repository during the build, I am not sure if this can be avoided though.
mvn install installs in the local repository
mvn deploy installs to the remote repository
these semantics are defined in the lifecycle and map to different plugins. Their implementations are different.
You don't have to manage the local repository. Actually for some if not most jobs you might even want to define it localized to the job (with the 'Use private Maven repository' option) instead of to the user who is running the job, especially that you plan to use nexus for repository.
You will have to change your jobs to use mvn deploy instead.
How can I make sure we don't run out of disk space on the server very
very quickly all the time?
Configure Jenkins/Nexus. Discard old builds and disable automatic artifact archiving. Both settings can be found in the Jenkins job-configuration. Also you could delete old artifacts automatically from Nexus using Scheduling Tasks.
There is no need to install the artifacts into the local maven repository when using Jenkins/Nexus on a dedicated server.
I am tying to set up a Jenkins Pipeline.
The first stage is done, the code compiles, is tested, inspected and deployed to Nexus.
I would like now to make a second stage on the pipeline where the war is checked out from Nexus and deployed on tomcat.
Actually I already integrated the maven-tomcat plugin to deploy on Tomcat.
My question is how can I check out the latest build of the war ?
Is there any maven or jenkins plugin for that ?
Many thanks,
Patrick
Your binary repository manager (Nexus) should ideally occupy the following position in you overall architecture:
You can use Jenkins as your provisioning tool, but ideally it should launch some sort of process which pulls the artifact to be deployed directly from Nexus (If nothing else it's more efficient).
This is a lot easier than it sounds. For example the Nexus REST API could be called from a shell script to download any desired revision of an artifact. For example:
$CATALINA_HOME/bin/shutdown.sh
curl -o $CATALINA_HOME/webapps/myfile.war http://myrepo.com/service/local/artifact/maven/redirect?r=releases&g=com.myorg&a=myfile&v=1.1.1&e=war
$CATALINA_HOME/bin/startup.sh
Finally, perhaps you might wish to consider a dedicated system for managing your deployments? An interesting solution I've been playing with is rundeck, which has a plugin for Jenkins. I really like rundeck, due to it's simplicity a trait it shares with Jenkins. There is also a plugin for Nexus that enables rundeck to provide a pull down list of artfacts eligible for deployment.
See download-artifact-from-nexus.sh script at https://github.com/cescoffier/puppet-nexus/tree/master/files
In my case, I modified it to use wget instead of curl. For some reason, curl wouldn't work for me.
I suggest you create a new pom for this. That way you are not bound to jenkins.
You need not explicitly checkout the artifact from nexus (note that this is called downloading from the repository in maven speech). You can specify a different war file location in the tomcat maven plugin. See the documentation. For downloading the latest version from the repository see the answers to this question.
i get the same problem with curl, i solved it buy adding the parameter -L, so that curl will follow the redirection to download the artifact, wget follows the redirection by default.
Below syntax has worked for me.
wget --user=admin --password=admin http://192.168.0.3:8081/repository/simpleapp-snapshot/in/javahome/simple-app/3.0.0-SNAPSHOT/simple-app-3.0.0-20210513.143540-1.war
I'm trying to setup a complete CI server, but I struggle on some points.
Currently, my system work as follow :
I commit local changes on my local GIT repository, then push to the GIT repository on the CI server
I then have a jenkins job triggered by the SCM change, who run a clean install and by doing so executes all my Junit and Jstestdriver test (via a local jstd server). This job deploy the snapshot artifact to a repository on my nexus repository
I installed M2-release-plugin for jenkins, and setup my pom.xml accordingly using maven-release-plugin. When i click on "Perform maven release" in jenkins job page, jenkins run mvn release:prepare release:perform, thus creating a tag in my git repo (say v000001) and deploying a versionned artifact on my nexus repository.
I don't really know if this process is fine, but i guess so...
My problem is that I want to deliver the versionned artifact in my nexus repo (say "artifact-v0000001.war") in my production tomcat. But I can't figure out how to do it.
When I do "mvn release:prepare release:perform tomcat:deploy" it deploys the new SNAPSHOT artifact built ... I don't want to do this, I want to reuse the artifact from the nexus repository.
Is there a way to doing this using a tool (maven/jenkins plugin, or external)?
Basically, I want to fetch the last release artifact on the repository, and send it to the tomcat manager for dereploying the webapp.
Do I need to setup a delivery job separated from the release job?
Jenkins, especially when combined with a tool like ANT, can do just about anything. It has a lot of plug-ins, and you can always write a script and incorporate it into a Jenkins build. Currently, I use Jenkins to deploy web applications to Windows IIS servers. What you could do here is have a Jenkins build that has your SVN path set in the source control section so that it fetches the latest version when you trigger the build. From there it should be fairly trivial to write an ANT script that copies it over the existing JAR in Tomcat, which will automatically restart it.
Your problem is that you are probably using the Jenkins release plugin, not the "m2 release plugin". The problem with the standard plugin is that it performs the regular build, saves the artifacts, then performs the release. It will then try to deploy the wrong artifacts that it created from the regular build.
The m2 release plugin solves this particular problem. There are some tricky workaround for this problem, but that's how it stands at the moment until this feature is implemented: https://issues.jenkins-ci.org/browse/JENKINS-11120 (log in and vote for it!)