I have looked up various Q&A's on how to use the artifactory features offered by GitLab, and they all talk about either JFrog (a.k.a Artifactory) or Nexus or something else. GitLab says that we could use it as Artifact management, and I was able to generate artifacts. But I am stuck now since I am completely lost on how to use this.
Here is the script I use to generate artifact using Gradle:
artifacts:
paths:
- build/libs/*.jar
Can anyone guide me on how I could refer to an artifact in my repo during CI/CD. Thanks in advance.
You can read up on them in the dependencies section of the Gitlab docs. Some quotes to guide you:
Note that artifacts from all previous stages are passed by default.
So every artifact you build in a prior job of the same pipeline is passed by default.
(...) the artifacts from job:x will be downloaded and extracted in
the context of the build.
In the context of the build means relative to the working path of the job.
To test this, add a job after the one you create the artifact in and add a ls -la command in the next job to see what has been extracted and is ready for use.
Related
In our project, we are using maven as the build tool, GitHub actions as the build agent, and Jfrog as the artifactory to deploy the jar files. We have both snapshot and release versions (identified using the "SNAPSHOT" in the version tag in pom). Currently we use the maven deploy plugin to deploy the built artifacts to Jfrog and the details of the repository are configured in the maven settings using the following tags in the maven profile:
When running maven deploy, the deploy plugin builds the project and deploys it to the artifactory and respective repository based on the artifact's version (has "SNAPSHOT" or not). After going through a few documents and blogs such as the following one:
https://jfrog.com/blog/dont-let-maven-deploy-plugin-trip-you
we thought it was better to use the jfrog artifactory plugin to achieve this so that we can capture build info and all among others. Since we are using GitHub actions, I could not find an artifactory plugin that is used for GitHub actions (found one for Jenkins and a few others but not for GitHub actions), I also don't want to add the plugin to my pom file and configure the repository details there as it would tie the repository details in pom. Also, I would like to separate the deploy logic from pom and move it to a CI server, so that these details can be hidden from the developers (the maven deploy plugin does this somewhat as all the artifactory config happens in the maven settings file)
The artifactory plugin can be configured in pom as follows:
referenced from https://www.jfrog.com/confluence/display/JFROG/Maven+Artifactory+Plugin
Then I found that we can use Jfrog CLI in GitHub actions to deploy the artifacts, but I could not find how I can configure the CLI to use both snapshot and release repositories so that I do not have to manually decide where to upload them by using the repo name. Can anybody guide me on how to achieve this in CLI?
I have referred to the following links from GitHub as well as jfrog:
https://github.com/marketplace/actions/setup-jfrog-cli
https://github.com/marketplace/actions/jfrog-cli-for-artifactory
https://jfrog.com/blog/jfrog-cli-github-actions-hero
https://jfrog.com/blog/publishing-binaries-using-the-jfrog-cli
https://jfrog.com/blog/using-the-jfrog-cli-with-github-actions
https://www.jfrog.com/confluence/display/CLI/JFrog+CLI
SO, I have finally managed to figure out how this can be done using github actions.
Im my actions.yml file, I have added a new step which makes use of "jfrog/setup-jfrog-cli" action from the github actions marketplace, the link to this is as follows. It shows you how to configure the action.
https://github.com/marketplace/actions/setup-jfrog-cli
I have configured the artifactory platform url as well as the username and access token in this action.
A new action was created which would build my code using the "jf maven" goal. Before you can make use of maven using jfrog CLI, it needs to be configured using the "jf mvn-config" command, it is here that you can specify which are the release and snapshot repositories that you want to make use of. The following is the code snippet that I have used for the same. Please note that these are steps within the github actions pipeline job and not the completed build yml file'
- name: Setup jfrog
uses: jfrog/setup-jfrog-cli#v3
with:
version: latest
env:
JF_URL: "https://artifactory.com" #be mindful not to add the path /artifactory here as it will cause authentication issue
JF_USER: ${{ secrets.ARTIFACT_USER_ID }}
JF_ACCESS_TOKEN: ${{ secrets.ARTIFACT_TOKEN }} # You have an option of giving password as well
- name: maven build
run: |
jf mvn-config --repo-deploy-releases=${ARTIFACTORY_RELEASE} --repo-deploy-snapshots=${ARTIFACTORY_SNAPSHOT} # mention the names of your release and snapshot repos within the jfrog artifacory
jf mvn deploy
jf rt bp # (optional, deploy build info, use --dry-run flag to see the build info in console without committing it)
There are many other optional parameters which you can use to enrich and configure the build, it can be found in the following links
https://www.jfrog.com/confluence/display/CLI/CLI+for+JFrog+Artifactory#CLIforJFrogArtifactory-RunningMavenBuilds
https://www.jfrog.com/confluence/display/JFROG/QuickStart+Guide%3A+Maven+and+Gradle
https://www.jfrog.com/confluence/display/CLI/CLI+for+JFrog+Artifactory#CLIforJFrogArtifactory-BuildIntegration
I am currently working on automating a Java Build and Deployment task where I am using GitHub Actions to perform the Build and Deployment. I am working on snapshots build and the artifact is pushed to JFrog Package Manager and the artifact name looks like jd-bulk-messenger-1.18.0-RC1-SNAPSHOT.war but when it gets pushed to my JFrog Snapshot Repository the same artifact seems to be uploaded with some timestamp as suffix like jd-bulk-messenger-1.18.0-RC1-20220715.124710-5.war.
How to ensure that I deploy the last recent snapshot artifacts on my target servers? I need some way to retrieve the last recent artifact name from JFrog Package Manager.
Note:
I know that we can use upload-artifact action to Archive the Artifact and can be used at later jobs with download-artifact action to perform the Deployment.
name: Archive Build Artifacts
uses: actions/upload-artifact#v3
But the Artifact size is of around 180 MB, so archiving the artifact of every build seems like not a good idea.
JFrog CLI offers extended support for complex download patterns. I think, following should help you -
First setup JFrog CLI in Github Action, check this link for installation.
Once CLI is configured to be used, you use following command to always download the latest snapshot.
jfrog rt dl --flat --limit=1 --sort-by=created --sort-order=desc --url=<ARTIFACTORY_URL> --user=<ARTIFACTORY_USER> --password=<ARTIFACTORY_PW> "<RELATIVE_PATH_TO_ARTIFACTORY_REPO>/jd-bulk-messenger-1.18.0-RC1-*.war"
You can further tune the command according to your need. Check this link to understand more.
Also yes 180MB is big to cache in CI build.
We are using Jenkins to build (maven) & deploy artifacts (JARs & *WAR*s) to an in-house artifactory server (both snapshots and releases).
For deployment, currently, we got Jenkins jobs that package the war file (from a release scm tag) and deploy to different environments/servers. We want to skip the package phase as it seems unnecessary to package it again & again for a released version because it's not possible to get a different copy of war file even after trying 1000 times.
We are looking for a way in Jenkins to get the artifact (war) from Artifactory and deploy it to a container. I am sure other people would have faced this situation too but I am not able to find any online material regarding this.
Is there any Jenkins plugin that takes a war file from Artifactory (based on a version) and deploy it to a remote container?
If this is not the right way of doing it then what are the recommendations for any other approach?
Thanks
I don't know about a plugin which takes a version # and deploys that, but you can build a Jenkins job to deploy the last successful release to a previous environment (thus copying from DEV-->QA for example.)
To do this, you would use the copy-artifact-plugin.
Here's an easy to follow run-through of this kind of setup:
http://www.lordofthejars.com/2012/09/deploying-jee-artifacts-with-jenkins.html
Every artifact stored in Artifactory will have a unique URL that includes the version number. It will take the format
http://artifactory-server/repository-name/path-to-artifact/version/filename
e.g.
http://artifactory/apps-releases-local/com/yourorg/yourapp/1.5.67/webapp.war
(depending on how you do your packaging, the WAR file name may include the version number as well).
So your deployment job can construct the Artifactory URL and download the file. Depending on how you have security set up in Artifactory, you may need to authenticate the request.
I have a maven multi module project with several modules. I want to deploy them (mvn deploy) only if they all pass a full mvn install (which includes the tests).
Currently, I run a mvn install on the project. If all modules pass, I run mvn deploy to do the deployment. The problem I see is the waste of time calling mvn twice (even if I skip tests on the second run).
Does anyone have an idea on this?
EDIT: I have learned that using Artifactory as a repository manager and the maven-artifactory-plugin with your maven setup will add the atomic deploy behaviour to the mvn deploy command. See the Build Integration section in the Artifactory documentation.
[DISCLOSURE - I'm associated with JFrog. Artifactory creator.]
Take a look at the deployAtEnd parameter of Maven Deployment plugin: http://maven.apache.org/plugins/maven-deploy-plugin/deploy-mojo.html
This is a bit tricky. Maven is not atomic when it executes the build life-cycle. So a broken set of artifacts may end up in a repository.
One solution I know is Nexus Pro: http://www.sonatype.com/Products/Nexus-Professional/Features - it allows you to promote builds or define certain repos as staging. So only verified versions get promoted to be used. Maybe artifactory has something similar - I just don't know.
If that solution is too expensive you probably need to create a cleanup build or profile to remove artifacts that where already uploaded. My first guess would be to write a Maven plugin to use the the proxy remote API or maybe the maven features are already sufficient. But since deploy means update the meta-data xml files too I dont think there is a delete - not sure on this either.
I am tying to set up a Jenkins Pipeline.
The first stage is done, the code compiles, is tested, inspected and deployed to Nexus.
I would like now to make a second stage on the pipeline where the war is checked out from Nexus and deployed on tomcat.
Actually I already integrated the maven-tomcat plugin to deploy on Tomcat.
My question is how can I check out the latest build of the war ?
Is there any maven or jenkins plugin for that ?
Many thanks,
Patrick
Your binary repository manager (Nexus) should ideally occupy the following position in you overall architecture:
You can use Jenkins as your provisioning tool, but ideally it should launch some sort of process which pulls the artifact to be deployed directly from Nexus (If nothing else it's more efficient).
This is a lot easier than it sounds. For example the Nexus REST API could be called from a shell script to download any desired revision of an artifact. For example:
$CATALINA_HOME/bin/shutdown.sh
curl -o $CATALINA_HOME/webapps/myfile.war http://myrepo.com/service/local/artifact/maven/redirect?r=releases&g=com.myorg&a=myfile&v=1.1.1&e=war
$CATALINA_HOME/bin/startup.sh
Finally, perhaps you might wish to consider a dedicated system for managing your deployments? An interesting solution I've been playing with is rundeck, which has a plugin for Jenkins. I really like rundeck, due to it's simplicity a trait it shares with Jenkins. There is also a plugin for Nexus that enables rundeck to provide a pull down list of artfacts eligible for deployment.
See download-artifact-from-nexus.sh script at https://github.com/cescoffier/puppet-nexus/tree/master/files
In my case, I modified it to use wget instead of curl. For some reason, curl wouldn't work for me.
I suggest you create a new pom for this. That way you are not bound to jenkins.
You need not explicitly checkout the artifact from nexus (note that this is called downloading from the repository in maven speech). You can specify a different war file location in the tomcat maven plugin. See the documentation. For downloading the latest version from the repository see the answers to this question.
i get the same problem with curl, i solved it buy adding the parameter -L, so that curl will follow the redirection to download the artifact, wget follows the redirection by default.
Below syntax has worked for me.
wget --user=admin --password=admin http://192.168.0.3:8081/repository/simpleapp-snapshot/in/javahome/simple-app/3.0.0-SNAPSHOT/simple-app-3.0.0-20210513.143540-1.war