For the purpose of checking software licenses, I need to download all artifacts (source jars) of a particular group id and a particular version from the maven repo. There are many artifacts under this group id and many version too. So need to recursively find all artifacts under the groupid.
I tried this wget command but it doesn't seem to recurse into the directories below.
wget \
--accept-regex '.*-1.1.1-sources.jar$' \
--execute robots=off \
--recursive \
--no-parent \
--level=0 \
-U "Any User Agent" \
--reject '*.html*' \
https://repo1.maven.org/maven2/org/theGroupId/
Is it possible to do this with wget or any other tool?
(The artifacts are from third party software that runs as a standalone process. My code doesn't have direct maven dependencies with it.)
Related
I have a Spring Boot 2.x project that uses Gradle 7.x.
I'm assembling a distribution of the artifact/service in a zip / tar file using the built-in Spring Boot task(s) provided. There is no meta-data associated with this asset, nor any need to add anything else to it.
I would like to copy (or publish) this zip / tar file into Artifactory (using Gradle), but so far everything I see around that subject includes (1) the file itself (usually a jar), (2) module meta-data and (3) the POM file.
Is there a way to accomplish what I'm looking for?
Not exactly answers your question, but an easier approach would be to upload to Artifactory using the JFrog CLI:
$ curl -fL https://getcli.jfrog.io
$ ./jfrog rt upload \
--url="https://domain.tld" \
--user="some_user" \
--password="the_password" \
file_to_upload.zip path/within/artifactory/to/place/the/file/
For more information see Uploading Files.
I'm aware of artifactory (which is planned in the future), but it's requested to package the artifacts on our releases.
We currently do on a script:
printf "\nCopying artifacts...\n"
for artifact_dir in "$HOME"/.m2/repository/com/foo/{*-ear,*-ui,*-tool}; do
highest_version=$(find "${artifact_dir}"/* -maxdepth 1 -type d -printf "%f\n" | sort -V | tail -1)
artifact_name=$(basename "${artifact_dir}")
mkdir --parent "${artifacts_out}/com/foo/${artifact_name}/${highest_version}"
cp --archive "${artifact_dir}/${highest_version}"/* \
"${artifacts_out}/com/foo/${artifact_name}/${highest_version}"
done
But as you can see it's not very neat, p.e we can not create a release with older artifacts because the script always takes the newest one.
Is there a maven plugin or something that permits to archive specific artifacts ?
If you want to create bundles of artifacts, run the Maven assembly plugin during the build. It can be used to gather artifacts, zip them and deploy the result to Artifactory/Nexus.
I have a multi-module maven project running in Jenkins. I would like to deploy the final artifact (an RPM from an assembly build) to the Nexus server. I see no reason to deploy intermediate artifacts (hence no "mvn clean deploy") since this will produce extra junk on the server that I don't need. We're trying to set up a continuous delivery pipeline, so we're not deploying SNAPSHOT versions - ever. The various plugins for Jenkins seem focused on deploying all of the artifacts. How can I just deploy the one I choose?
EDIT:
After much consideration, I'd be willing to deploy all the artifacts to nexus if the deploy happens after all of the build has completed. If I was going that route, I'd want to use the "Deploy artifacts to maven repository" post build action. This tool seems broken though because it's missing the functionality of specifying the username/password. I know this can be specified in the settings.xml, but apparently only the one in ".m2". It doesn't seem to be looking at the settings.xml I specified for this build.
This also seems like it's broken for redeploying artifacts. There is some verbage in "Jenkins the Definitive Guide" for this, but it talks about a "local" settings.xml. All my builds are happening on Jenkins slaves, so this isn't an option and doesn't even really make sense with the Jenkins architecture.
If you need still generic way.
Use Execute shell option and use the mvn deploy command manually, You can pass your version and others things such as groupID etc as parameters in job, If you maintain separate job to build and upload this will work out.
Ex:
export M2_HOME=/PATH/TO/softwares/apache-maven-3.0.4
PATH=$M2_HOME/bin:$JAVA_HOME/bin:$PATH
export PATH
mvn -v
mvn deploy:deploy-file -Durl=http://someorg:8081/nexus/content/repositories/t1.snapshot/ -DrepositoryId=t1.snapshot -DartifactId=artifactID -DgroupId=groupID -Dpackaging=zip -Dfile=${WORKSPACE}/filename.zip -Dversion=1.0-TEST-SNAPSHOT -s "/path/to/.m2/settings.xml"
You can make use of 'Deploy artifacts to Maven Repository' under 'Post-Build Actions'. Take look at this answer
Here's the ugly hack I came up with. I'll gladly give someone else the "correct" answer for this if anyone has a better idea:
I realized that I need to deploy both the parent pom.xml and the assembly. I did this in two separate post build steps.
First, I chose "Invoke top-level Maven targets" with a Maven Version of "Maven" (I think this uses Jenkins version of maven. I don't want to put a different version on the system). I used Goals of:
-s svn-admin/settings.xml -N deploy
That deploys just the parent pom to nexus with my specified settings.xml.
The REALLY big hack happens when I want to deploy the rpm. I tried a "deploy-file" target, but without a variable I could expand to the version number, I couldn't specify the exact file and wildcards don't expand. Instead I did an "Execute shell" option and used curl I found here:
env
UPLOAD_FILE=assembly/target/ips-${POM_VERSION}-x.x86_64.rpm
DESTINATION=http://mvnrepo01/nexus/content/repositories/releases/com/bla/ips/assembly/${POM_VERSION}/assembly-${POM_VERSION}.rpm
sha1sum ${UPLOAD_FILE} | awk -F" " '{print $1}' | curl -v -u admin:password --upload-file - ${DESTINATION}.sha1
md5sum ${UPLOAD_FILE} | awk -F" " '{print $1}' | curl -v -u admin:password --upload-file - ${DESTINATION}.md5
curl -v -u admin:password --upload-file ${UPLOAD_FILE} ${DESTINATION}
UPLOAD_FILE=assembly/pom.xml
DESTINATION=http://mvnrepo01/nexus/content/repositories/releases/com/bla/ips/assembly/${POM_VERSION}/assembly-${POM_VERSION}.pom
sha1sum ${UPLOAD_FILE} | awk -F" " '{print $1}' | curl -v -u admin:password --upload-file - ${DESTINATION}.sha1
md5sum ${UPLOAD_FILE} | awk -F" " '{print $1}' | curl -v -u admin:password --upload-file - ${DESTINATION}.md5
curl -v -u admin:password --upload-file ${UPLOAD_FILE} ${DESTINATION}
Like i said, this is an ugly hack. I'm pretty sure there are metadata files that aren't getting updated, but the rpm, it's pom, and their checksums are getting uploaded.
You can use Nexus Jenkins Plugin to deploy a specific artifact from Jenkins into Nexus:
https://support.sonatype.com/hc/en-us/articles/227256688-How-do-I-configure-the-Nexus-Jenkins-Plugin
Example of Jenkins pipeline:
stage('Publish') {
def pom = readMavenPom file: 'pom.xml'
nexusPublisher nexusInstanceId: 'your-nexus-instance-id', \
nexusRepositoryId: 'your-nexus-repository-id', \
packages: [[$class: 'MavenPackage', \
mavenAssetList: [[classifier: '', extension: '', filePath: "target/${pom.artifactId}-${pom.version}.${pom.packaging}"]], \
mavenCoordinate: [artifactId: "${pom.artifactId}", \
groupId: "${pom.groupId}", \
packaging: "${pom.packaging}", \
version: "${pom.version}"]]]
}
In this case Nexus Jenkins Plugin will deploy only your target/${pom.artifactId}-${pom.version}.${pom.packaging} and pom.xml files to Nexus repository.
I think I have run in to a maven deploy plugin bug, but I want to confirm that I am not doing it wrong before I open a bug report and start grepping through source.
I am trying to deploy an arbitrary binary (tar.gz,jar,box,etc) to a maven repository.
The user I am using is in the settings.xml file with a plaintext password.
The user is in an LDAP group that is connected to nexus with a Role.
That role has 3 privileges associated with it. Update, Create, Read which are in turn associated with the repository target.
Now using wget the user can pull data from the repo, and using this curl command I can deploy to the repo:
curl --request PUT --user USERNAME:PASSWORD https://NEXUS_URL/nexus/content/repositories/REPO_NAME/path/to/file/with/groupid/version/iamafile.jar \
--data #./iamafile.jar -H Content-Type:application/java-archive --verbose
This works fine, http 201 and the file is added. When I use a similiar mvn deploy command:
mvn deploy:deploy-file --settings /path/to/settings.xml \
-Durl=https://NEXUS_URL/nexus/content/repositories/REPOSITORYID -Dfile=iamafile.jar \
-DrepositoryId=REPOSITORYID -DartifactId=ARTIFACTID -DgroupId=GROUPID \
-Dversion=VERSION
I get an http 401 unauthorized error.
There have been multiple bug reports about this, but they supposedly have been fixed.
With a pom file with the same values for version, artifcatId, etc. mvn deploy works fine.
Have I found a bug, or am I using it improperly?
Edit:
I opened up this bug regarding the issue.
If the dev's end up saying that it isn't a bug, I will add that as an answer if someone hasn't beat me to it.
you need to additionally specify the -DrepositoryId
Actually, I want to integrate spring javadoc with my netbeans IDE. Spring website only provides an HTML version javadoc online, no download link. But Netbeans only accept a jar/zip file or a local folder.
I know I can build it myself from spring source, but would rather not go through the work.
You can download from this location http://repo.spring.io/libs-release/org/springframework/spring/
You need the -with-docs.zip download
http://www.springsource.com/download/community
It seems the downloads are now managed with Maven only, however, if you want a copy of the JavaDocs locally you could use wget on the JavaDoc URL.
e.g. For release 3.0.5.RELEASE this works a treat.
wget \
--recursive \
--no-clobber \
--page-requisites \
--html-extension \
--convert-links \
--restrict-file-names=windows \
--domains docs.spring.io \
--no-parent \
http://docs.spring.io/spring/docs/3.0.5.RELEASE/javadoc-api/
Or you can just grab the version with source code (which is even better in your IDE anyway).
Sign up over at http://www.springsource.com or just grab it from a maven repository like this one:
http://www.jarvana.com/jarvana/browse/org/springframework/spring-jdbc/
Just grab the release you want (like 2.5.6/spring-jdbc-2.5.6-sources.jar), save to disk, and in Netbeans right-click on your library and select "add local sources".
/ Papa Freud
Using of Maven works for me. In the directory with pom.xml type the command:
mvn dependency:sources -DincludeArtifactIds=spring-aop -Dclassifier=javadoc
After that the jar is available in the ~/.m2 directory.
Now Spring uses artifactory.
Go to corresponding folder for further download.
http://repo.spring.io/webapp/browserepo.html?24