How to store jars in github based maven repo - maven

Ive setup a github repo to store custom archetypes, and I can start a new project off these archetypes (on separate vm) using something like
mvn archetype:generate -Dcatalog=https://raw.github.etc./archetype-catalog.xml
now i'm trying to store some 3rd party jars remotely in the same repo and reference them from a pom.xml as dependancies
First I installed the jar locally with
mvn install:install-file ..params..
the repo structure I have goes like:
https://github.com/myappco/myrepo
|
|___releases
| |
(group id)adobe.flex.messaging (ie. adobe/flex/messaging)
common
1.0
(common-1.0.jar)
* trying to make this work
|___snapshots
|
(group id)com.acme.archetypes (ie. com/acme/archetypes)
my-archetype
1.0-SNAPSHOT
* archetype resolved ok
archetype-catalog.xml
The settings in my project:
<dependency>
<groupId>adobe.flex.messaging</groupId>
<artifactId>common</artifactId>
<version>1.0</version>
<scope>test</scope>
</dependency>
<repositories>
<repository>
<id>orb-snapshots</id>
<url>https://github.com/myappco/myrepo/releases</url>
</repository>
</repositories>
I think I must be doing something dumb, because I can archetype:generate from the github repo archetype-catalog.xml. Can anyone see what I messed up?
Ta

You need to deploy your artifacts in github using mvn deploy. You would need to have set up github appropriately - perhaps similar to what is mentioned in Hosting Maven Repos on Github.

Related

Munit in Jenkins with dependency on Domain projects

I have Mule Project "A" which refers a Domain Project "X". I would like to run my MUnits for Project "A" in Jenkins but it fails as it's unable to find the dependency Domain project "X".
To resolve this I added the "dependency" for domain project"A" in, project "X" pom. xml. But no luck.
Please give me details how it can be resolved to run my MUnits ion Jenkins as a post-build action.
Thanks,
Ikram
to be able to do that I had to:
First, add the domain dependency in your pom with "test" scope
<dependency>
<groupId>com.mygroupid</groupId>
<artifactId>mule-domain-id</artifactId>
<version>1.0.0</version>
<scope>test</scope>
<type>zip</type>
</dependency>
Second, you have to install the domain artifact in one artifact repository (Nexus for example).This way, the dependency is accessible from Jenkins. And then add that repository to your pom.xml
<repositories>
<repository>
<id>alm-nexus</id>
<name>ALM Nexus Repo</name>
<url>https://nexus.alm.corp/repository/maven-public</url>
<layout>default</layout>
</repository>
</repositories>
One important thing when runing MUnits in Jenkins is the following. The root folder name has to be exactly the same as the artifactId, so in Jenkins you have to set the workspace with a custom one
If using a Jenkinsfile you can do this with the following:
customWorkspace
"${env.JOB_NAME}/${env.BUILD_ID}/artifact-id-has-to-go-here"
With these steps we are able to have a working pipeline with MUnit testing as one of the stages.

How can I add maven artifact into an existing maven project

How can I add maven artifact into an existing maven project.I understand that I can build a jar locally and use file: protocol but this is possible using maven also.
For example I have a basic maven project
https://maven.apache.org/guides/getting-started/index.html#How_do_I_make_my_first_Maven_project and the artifact:
<repositories>
<repository>
<id>myrepo.org</id>
<name>MyRepository</name>
<url>http://mywork.com/repository</url>
</repository>
</repositories>
<dependency>
<groupId>org.ethereum</groupId>
<artifactId>ethereumj-core</artifactId>
<version>1.1.0-RELEASE</version>
I tried adding the code above to the project pom.xml fails because dependency is not in central maven repo.
mvn clean install
I then tried editing my settings.xml by adding the tag, that also failed because dependency was not found.
Im overlooking something pretty basic here.
Maven works on the concept of local and remote repositories.
The local repository refers to a copy on your own installation that is a cache of the remote downloads, and also contains the temporary build artifacts that you have not yet released.
Remote repository is repository you access the artifacts via file or http / ftp protocols , it can be an internal repo or a remote public hosted.
When you add dependency maven search that artifact in local if not found then remote repo will be searched. Still not found then error is reported.
https://maven.apache.org/guides/introduction/introduction-to-repositories.html
In your case, 'ethereumj-core' can not be located neither of location, you need find this jar and do a manual install to local repo.
mvn install:install-file -Dfile=< folder >\ethereumj-core.1.1.0-RELEASE.jar -DgroupId=org.ethereum
-DartifactId=ethereumj-core -Dversion=1.1.0-RELEASE -Dpackaging=jar
once properly installed maven should be able find this artifact when you add this as dependency to any of the projects (in pom.xml).
<dependency>
<groupId>org.ethereum</groupId>
<artifactId>ethereumj-core</artifactId>
<version>1.1.0-RELEASE</version>
</dependency>

Adding db2 jars to java webapp using maven

I'm trying to add the following db2 jars to my Java web application using Maven...
db2jcc_license_cu.jar
db2jcc_javax.jar
db2jcc.jar
I'm following the instructions posted in this post...
Can I add jars to maven 2 build classpath without installing them?
I want to use the static in-project repository solution. So far I have...
Created a folder in my root directory named lib. Inside this
directory lives the three db2 jars.
Added the following to my pom file...
<repository>
<id>lib</id>
<releases>
<enabled>true</enabled>
<checksumPolicy>ignore</checksumPolicy>
</releases>
<snapshots>
<enabled>false</enabled>
</snapshots>
<url>file://${project.basedir}/lib</url>
</repository>
</repositories>
<dependency>
<groupId>com.ibm.db2.jcc</groupId>
<artifactId>db2jcc</artifactId>
<version>3.8.47</version>
</dependency>
<dependency>
<groupId>com.ibm.db2.jcc</groupId>
<artifactId>db2jcc_license_cu</artifactId>
<version>3.8.47</version>
</dependency>
But when I run a maven install I get ...
[WARNING] The POM for com.ibm.db2.jcc:db2jcc:jar:3.8.47 is missing, no dependency information available
[WARNING] The POM for com.ibm.db2.jcc:db2jcc_license_cu:jar:3.8.47 is missing, no dependency information available
I got the version of the Jars by running a...
java com.ibm.db2.jcc.DB2Jcc -version
Have I specified this version info corretly? Can anyone suggest what I am doing wrong?
The problem is that you didn't install the jars properly in your "project-maven-repository" (i.e. in the folder ${project.basedir}/lib)
Maven stores (when you do mvn install) the jar files in a maven repository. A maven repository have precise hierarchical structure. Here is a simplified vision of this structure:
the artifact groupId+artifactId define the first part of folder path (in the repository) where the artifact is stored.
the artifact version is the second part of the folder path
the artifact version is also a suffix to the artifact name
the artifactId is the artifact name
the packaging is the artifact extension (default is jar)
By default maven use a repository located under <USER_HOME>/.m2/repository
The solution you are trying to setup use another location for the repository : ${project.basedir}/lib and even if it is not the default repository location it is still a maven-repository and so maven is expecting to find the usual maven repository hierarchy under this location.
That's why you need to organize your ${project.basedir}/lib folder just like a maven repository. That's explained in this part of the referenced post:
Use Maven to install to project repo
Instead of creating this structure by hand I recommend to use a Maven plugin to install your jars as artifacts. So, to install an artifact to an in-project repository under repo folder execute:
mvn install:install-file -DlocalRepositoryPath=lib -DcreateChecksum=true -Dpackaging=jar -Dfile=[your-jar] -DgroupId=[...] -DartifactId=[...] -Dversion=[...]
If you'll choose this approach you'll be able to simplify the repository declaration in pom to:
<repository>
<id>repo</id>
<url>file://${project.basedir}/lib</url>
</repository>
So you need to do an mvn install to create the ${project.basedir}/lib hierarchy (you can do it by hand, but it's not recommended and error prone).
I your case, the commands to run will be like this: (assuming you put the jar in your HOME_DIR and run this command in your ${project.basedir})
mvn install:install-file -DlocalRepositoryPath=lib -DcreateChecksum=true -Dpackaging=jar -Dfile=<USER_HOME>/db2jcc_license_cu.jar -DgroupId=com.ibm.db2.jcc -DartifactId=db2jcc_license_cu -Dversion=3.8.47
What are the advantages of the approch you choose :
a developer with no maven setup will have the libraries available inside the project sources, under SCM system.
you can easily reference jars that aren't in a public maven repository without the need of something like artifactory or nexus
The drawbacks :
a quite complex folder structure under ${project.basedir}/lib looking very strange for someone not used to work with maven.
you will store the libraries under SCM (lot's of huge binary files)
Another solution would be to download those jars before hand and put them somewhere relatively to your project (like lib directory). Now just tell maven to use those jars. Here the groupId, artifactdId and version are JFYI since they won't be used to download anything.
The merit of this solution is that you won't have to build a maven repository.
<dependencies>
...
<dependency>
<groupId>com.ibm.db2.jcc</groupId>
<artifactId>licences</artifactId>
<version>1.0</version> <!-- Adjust this properly -->
<scope>system</scope>
<systemPath>${basedir}/lib/db2jcc_license_cu.jar</systemPath>
</dependency>
<dependency>
<groupId>com.ibm.db2.jcc</groupId>
<artifactId>db2jcc4</artifactId>
<version>1.0</version> <!-- Adjust this properly -->
<scope>system</scope>
<systemPath>${basedir}/lib/db2jcc4.jar</systemPath>
</dependency>
<dependency>
<groupId>com.ibm.db2.jcc</groupId>
<artifactId>db2jcc_javax</artifactId>
<version>1.0</version> <!-- Adjust this properly -->
<scope>system</scope>
<systemPath>${basedir}/lib/db2jcc_javax.jar</systemPath>
</dependency>
</dependencies>
Refer Link (Japanese): Mavenリポジトリで提供されていないサードパーティJarをどうするか
I guess these jars do not have a pom.xml. Hence the warning. If the jars get packaged and the application works, then I guess you do not have a problem.

Hosting a Maven repository on github

I have a fork of a small open sourced library that I'm working on github. I'd like to make it available to other developers via maven, but I don't want to run my own Nexus server, and because it's a fork I can't easily deploy it to oss.sonatype.org.
What I'd like to do is to deploy it to github so that others can access it using maven. What's the best way to do this?
The best solution I've been able to find consists of these steps:
Create a branch called mvn-repo to host your maven artifacts.
Use the github site-maven-plugin to push your artifacts to github.
Configure maven to use your remote mvn-repo as a maven repository.
There are several benefits to using this approach:
Maven artifacts are kept separate from your source in a separate branch called mvn-repo, much like github pages are kept in a separate branch called gh-pages (if you use github pages)
Unlike some other proposed solutions, it doesn't conflict with your gh-pages if you're using them.
Ties in naturally with the deploy target so there are no new maven commands to learn. Just use mvn deploy as you normally would
The typical way you deploy artifacts to a remote maven repo is to use mvn deploy, so let's patch into that mechanism for this solution.
First, tell maven to deploy artifacts to a temporary staging location inside your target directory. Add this to your pom.xml:
<distributionManagement>
<repository>
<id>internal.repo</id>
<name>Temporary Staging Repository</name>
<url>file://${project.build.directory}/mvn-repo</url>
</repository>
</distributionManagement>
<plugins>
<plugin>
<artifactId>maven-deploy-plugin</artifactId>
<version>2.8.1</version>
<configuration>
<altDeploymentRepository>internal.repo::default::file://${project.build.directory}/mvn-repo</altDeploymentRepository>
</configuration>
</plugin>
</plugins>
Now try running mvn clean deploy. You'll see that it deployed your maven repository to target/mvn-repo. The next step is to get it to upload that directory to GitHub.
Add your authentication information to ~/.m2/settings.xml so that the github site-maven-plugin can push to GitHub:
<!-- NOTE: MAKE SURE THAT settings.xml IS NOT WORLD READABLE! -->
<settings>
<servers>
<server>
<id>github</id>
<username>YOUR-USERNAME</username>
<password>YOUR-PASSWORD</password>
</server>
</servers>
</settings>
(As noted, please make sure to chmod 700 settings.xml to ensure no one can read your password in the file. If someone knows how to make site-maven-plugin prompt for a password instead of requiring it in a config file, let me know.)
Then tell the GitHub site-maven-plugin about the new server you just configured by adding the following to your pom:
<properties>
<!-- github server corresponds to entry in ~/.m2/settings.xml -->
<github.global.server>github</github.global.server>
</properties>
Finally, configure the site-maven-plugin to upload from your temporary staging repo to your mvn-repo branch on Github:
<build>
<plugins>
<plugin>
<groupId>com.github.github</groupId>
<artifactId>site-maven-plugin</artifactId>
<version>0.11</version>
<configuration>
<message>Maven artifacts for ${project.version}</message> <!-- git commit message -->
<noJekyll>true</noJekyll> <!-- disable webpage processing -->
<outputDirectory>${project.build.directory}/mvn-repo</outputDirectory> <!-- matches distribution management repository url above -->
<branch>refs/heads/mvn-repo</branch> <!-- remote branch name -->
<includes><include>**/*</include></includes>
<repositoryName>YOUR-REPOSITORY-NAME</repositoryName> <!-- github repo name -->
<repositoryOwner>YOUR-GITHUB-USERNAME</repositoryOwner> <!-- github username -->
</configuration>
<executions>
<!-- run site-maven-plugin's 'site' target as part of the build's normal 'deploy' phase -->
<execution>
<goals>
<goal>site</goal>
</goals>
<phase>deploy</phase>
</execution>
</executions>
</plugin>
</plugins>
</build>
The mvn-repo branch does not need to exist, it will be created for you.
Now run mvn clean deploy again. You should see maven-deploy-plugin "upload" the files to your local staging repository in the target directory, then site-maven-plugin committing those files and pushing them to the server.
[INFO] Scanning for projects...
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] Building DaoCore 1.3-SNAPSHOT
[INFO] ------------------------------------------------------------------------
...
[INFO] --- maven-deploy-plugin:2.5:deploy (default-deploy) # greendao ---
Uploaded: file:///Users/mike/Projects/greendao-emmby/DaoCore/target/mvn-repo/com/greendao-orm/greendao/1.3-SNAPSHOT/greendao-1.3-20121223.182256-3.jar (77 KB at 2936.9 KB/sec)
Uploaded: file:///Users/mike/Projects/greendao-emmby/DaoCore/target/mvn-repo/com/greendao-orm/greendao/1.3-SNAPSHOT/greendao-1.3-20121223.182256-3.pom (3 KB at 1402.3 KB/sec)
Uploaded: file:///Users/mike/Projects/greendao-emmby/DaoCore/target/mvn-repo/com/greendao-orm/greendao/1.3-SNAPSHOT/maven-metadata.xml (768 B at 150.0 KB/sec)
Uploaded: file:///Users/mike/Projects/greendao-emmby/DaoCore/target/mvn-repo/com/greendao-orm/greendao/maven-metadata.xml (282 B at 91.8 KB/sec)
[INFO]
[INFO] --- site-maven-plugin:0.7:site (default) # greendao ---
[INFO] Creating 24 blobs
[INFO] Creating tree with 25 blob entries
[INFO] Creating commit with SHA-1: 0b8444e487a8acf9caabe7ec18a4e9cff4964809
[INFO] Updating reference refs/heads/mvn-repo from ab7afb9a228bf33d9e04db39d178f96a7a225593 to 0b8444e487a8acf9caabe7ec18a4e9cff4964809
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 8.595s
[INFO] Finished at: Sun Dec 23 11:23:03 MST 2012
[INFO] Final Memory: 9M/81M
[INFO] ------------------------------------------------------------------------
Visit github.com in your browser, select the mvn-repo branch, and verify that all your binaries are now there.
Congratulations!
You can now deploy your maven artifacts to a poor man's public repo simply by running mvn clean deploy.
There's one more step you'll want to take, which is to configure any poms that depend on your pom to know where your repository is. Add the following snippet to any project's pom that depends on your project:
<repositories>
<repository>
<id>YOUR-PROJECT-NAME-mvn-repo</id>
<url>https://github.com/YOUR-USERNAME/YOUR-PROJECT-NAME/raw/mvn-repo/</url>
<snapshots>
<enabled>true</enabled>
<updatePolicy>always</updatePolicy>
</snapshots>
</repository>
</repositories>
Now any project that requires your jar files will automatically download them from your github maven repository.
Edit: to avoid the problem mentioned in the comments ('Error creating commit: Invalid request. For 'properties/name', nil is not a string.'), make sure you state a name in your profile on github.
Don't use GitHub as a Maven Repository.
Edit: This option gets a lot of down votes, but no comments as to why. This is the correct option regardless of the technical capabilities to actually host on GitHub. Hosting on GitHub is wrong for all the reasons outlined below and without comments I can't improve the answer to clarify your issues.
Best Option - Collaborate with the Original Project
The best option is to convince the original project to include your changes and stick with the original.
Alternative - Maintain your own Fork
Since you have forked an open source library, and your fork is also open source, you can upload your fork to Maven Central (read Guide to uploading artifacts to the Central Repository) by giving it a new groupId and maybe a new artifactId.
Only consider this option if you are willing to maintain this fork until the changes are incorporated into the original project and then you should abandon this one.
Really consider hard whether a fork is the right option. Read the myriad Google results for 'why not to fork'
Reasoning
Bloating your repository with jars increases download size for no benefit
A jar is an output of your project, it can be regenerated at any time from its inputs, and your GitHub repo should contain only inputs.
Don't believe me? Then check Google results for 'dont store binaries in git'.
GitHub's help Working with large files will tell you the same thing. Admittedly jar's aren't large but they are larger than the source code and once a jar has been created by a release they have no reason to be versioned - that is what a new release is for.
Defining multiple repos in your pom.xml slows your build down by Number of Repositories times Number of Artifacts
Stephen Connolly says:
If anyone adds your repo they impact their build performance
as they now have another repo to check artifacts against... It's not a big
problem if you only have to add one repo... But the problem grows and the next
thing you know your maven build is checking 50 repos for every artifact and
build time is a dog.
That's right! Maven needs to check every artifact (and its dependencies) defined in your pom.xml against every Repository you have defined, as a newer version might be available in any of those repositories.
Try it out for yourself and you will feel the pain of a slow build.
The best place for artifacts is in Maven Central, as its the central place for jars, and this means your build will only ever check one place.
You can read some more about repositories at Maven's documentation on Introduction to Repositories
You can use JitPack (free for public Git repositories) to expose your GitHub repository as a Maven artifact. Its very easy. Your users would need to add this to their pom.xml:
Add repository:
<repository>
<id>jitpack.io</id>
<url>https://jitpack.io</url>
</repository>
Add dependency:
<dependency>
<groupId>com.github.User</groupId>
<artifactId>Repo name</artifactId>
<version>Release tag</version>
</dependency>
As answered elsewhere the idea is that JitPack will build your GitHub repo and will serve the jars. The requirement is that you have a build file and a GitHub release.
The nice thing is that you don't have to handle deployment and uploads. Since you didn't want to maintain your own artifact repository its a good match for your needs.
Since 2019 you can now use the new functionality called Github package registry.
Basically the process is:
generate a new personal access token from the github settings
add repository and token info in your settings.xml
deploy using
mvn deploy -Dregistry=https://maven.pkg.github.com/yourusername -Dtoken=yor_token
Another alternative is to use any web hosting with webdav support. You will need some space for this somewhere of course but it is straightforward to set up and a good alternative to running a full blown nexus server.
add this to your build section
<extensions>
<extension>
<artifactId>wagon-webdav-jackrabbit</artifactId>
<groupId>org.apache.maven.wagon</groupId>
<version>2.2</version>
</extension>
</extensions>
Add something like this to your distributionManagement section
<repository>
<id>release.repo</id>
<url>dav:http://repo.jillesvangurp.com/releases/</url>
</repository>
Finally make sure to setup the repository access in your settings.xml
add this to your servers section
<server>
<id>release.repo</id>
<username>xxxx</username>
<password>xxxx</password>
</server>
and a definition to your repositories section
<repository>
<id>release.repo</id>
<url>http://repo.jillesvangurp.com/releases</url>
<releases>
<enabled>true</enabled>
</releases>
<snapshots>
<enabled>false</enabled>
</snapshots>
</repository>
Finally, if you have any standard php hosting, you can use something like sabredav to add webdav capabilities.
Advantages: you have your own maven repository
Downsides: you don't have any of the management capabilities in nexus; you need some webdav setup somewhere
As an alternative, Bintray provides free hosting of maven repositories. That's probably a good alternative to Sonatype OSS and Maven Central if you absolutely don't want to rename the groupId. But please, at least make an effort to get your changes integrated upstream or rename and publish to Central. It makes it much easier for others to use your fork.
If you have only aar or jar file itself, or just don't want to use plugins - I've created a simple shell script. You can achieve the same with it - publishing your artifacts to Github and use it as public Maven repo.
I'd like to add another alternative, a Gradle plugin I've been working on lately: magik.
Basically it allows to publish directly on a github repository acting as a maven repository.
I came here looking to do the same thing, unlitmately host my Maven repository for free, but after more research I ended up here:
https://jfrog.com/start-free/
The setup was quite strightforward, has a good free tier which will serve me for the forseeable future, and has additional (paid for) upgrades which may well come in handy in the future.
So far I am very pleased indeed !

install JAR from remote repo (clojar)

I want to download and install this Clojure library but cannot for the life of me figure it out. I researched Maven, but couldn't get it to find the repo. How can I easily install a Clojure library onto my machine?
You could add the repository containing this jar to your pom or settings file and specify the relevant jar as a dependency.
<repository>
<id>clojars.org</id>
<url>http://clojars.org/repo</url>
</repository>
...
<dependency>
<groupId>clj-http</groupId>
<artifactId>clj-http</artifactId>
<version>0.4.1</version>
</dependency>
download https://clojars.org/repo/clj-http/clj-http/0.4.1/clj-http-0.4.1.jar
mvn install:install-file -DgroupId=clj-http -DartifactId=clj-http -Dversion=0.4.1 -Dpackaging=jar -Dfile=clj-http-0.4.1.jar
If you are using Leiningen you can just read all about how to connect to repository server on a recent Sonatype blog post from Tim O'Brien.
If you are using Maven you should get a repo server like Nexus and setup the clojure repo as another proxy repository and add it to your public group.
If neither of these approaches is ok with you you can use Raghurams approach or number23_cn. Both of them however are semi optimal and will not scale for teams or many artifacts..

Resources