The XADisk library deployed on Maven Central packaged as 'rar' instead of 'jar'. But i just need the jar (and possibly source) for the project i'm working on. I was wondering what the best (maven style) way is to deal with this dependency.
The jar files are available on Central but not specified in the pom thus type="jar" doesnt work
the pom is here: https://repo1.maven.org/maven2/net/java/xadisk/xadisk/1.2.2/xadisk-1.2.2.pom
and the jars can be found here: https://repo1.maven.org/maven2/net/java/xadisk/xadisk/1.2.2/xadisk-1.2.2.pom
I can't reproduce your issue, maybe they changed something in the repo?
If I add the following dependency to my project:
<dependency>
<groupId>net.java.xadisk</groupId>
<artifactId>xadisk</artifactId>
<version>1.2.2</version>
</dependency>
then the JAR file gets packaged into my project.
By the way, if no "type" is specified (for the dependency), maven uses JAR as default.
Related
I'm working on a recommender system using Apache Flink. The implementation is running when I test it in IntelliJ, but I would like now to go on a cluster. I also built a jar file and tested it locally to see if all was working but I encountered a problem.
java.lang.NoClassDefFoundError: org/apache/flink/ml/common/FlinkMLTools$
As we can see, the class FlinkMLTools used in my code isn't found during the running of the jar.
I built this jar with Maven 3.3.3 with mvn clean install and I'm using the version 0.9.0 of Flink.
First Trail
The fact is that my global project contains other projects (and this recommender is one of the sub-project). In that way, I have to launch the mvn clean install in the folder of the right project, otherwise Maven always builds a jar of an other project (and I don't understand why). So I'm wondering if there could be a way to say explicitly to maven to build one specific project of the global project. Indeed, perhaps the path to FlinkMLTools is contained in a link present in the pom.xml file of the global project.
Any other ideas?
The problem is that Flink's binary distribution does not contain the libraries (flink-ml, gelly, etc.). This means that you either have to ship the library jar files with your job jar or that you have to copy them manually to your cluster. I strongly recommend the first option.
Building a fat-jar to include library jars
The easiest way to build a fat jar which does not contain unnecessary jars is to use Flink's quickstart archetype to set up the project's pom.
mvn archetype:generate -DarchetypeGroupId=org.apache.flink \
-DarchetypeArtifactId=flink-quickstart-scala -DarchetypeVersion=0.9.0
will create the structure for a Flink project using the Scala API. The generated pom file will have the following dependencies.
<dependencies>
<dependency>
<groupId>org.apache.flink</groupId>
<artifactId>flink-scala</artifactId>
<version>0.9.0</version>
</dependency>
<dependency>
<groupId>org.apache.flink</groupId>
<artifactId>flink-streaming-scala</artifactId>
<version>0.9.0</version>
</dependency>
<dependency>
<groupId>org.apache.flink</groupId>
<artifactId>flink-clients</artifactId>
<version>0.9.0</version>
</dependency>
</dependencies>
You can remove flink-streaming-scala and instead you insert the following dependency tag in order to include Flink's machine learning library.
<dependency>
<groupId>org.apache.flink</groupId>
<artifactId>flink-ml</artifactId>
<version>0.9.0</version>
</dependency>
When you know build the job jar with mvn package, the generated jar should contain the flink-ml jar and all of its transitive dependencies.
Copying the library jars manually to the cluster
Flink includes all jars which are located in the <FLINK_ROOT_DIR>/lib folder in the classpath of the executed jobs. Thus, in order to use Flink's machine learning library you have to put the flink-ml jar and all needed transitive dependencies into the /lib folder. This is rather tricky, since you have to figure out which transitive dependencies are actually needed by your algorithm and, consequently, you will often end up copying all transitive dependencies.
How to build a specific sub-module with maven
In order to build a specific sub-module X from your parent project you can use the following command:
mvn clean package -pl X -am
-pl allows you to specify which sub-modules you want to build and -am tells maven to also build other required sub-modules. It is also described here.
In cluster mode, Flink does not put all library JAR files into the classpath of its workers. When executing the program locally in IntelliJ all required dependencies are in the classpath, but not when executing on a cluster.
You have two options:
copy the FlinkML Jar file into the lib folder of all Flink TaskManager
Build a fat Jar file for you application that includes the FLinkML dependencies.
See the Cluster Execution Documentation for details.
I've been working with Maven for a little while now and I had a question about the information shown on the Maven Repository site. I was looking at the tags to paste into my pom for spring-web-mvc 3.2.8.RELEASE and noticed the table with the header "this artifact depends on" and saw the host of artifacts listed below.
My question is simple: Am I supposed to include the all of the dependencies listed in that table in my pom?
To answer your question, no you do not need to include all of the dependencies listed in the artifact dependencies section. It is my understanding that when you include a dependency in your pom file, maven will automatically download any needed jars. I am inferring this due to the fact that I personally don't add any of the artifact's dependencies other than what I need to my pom.
For example if I wanted spring-core I would do the following:
<dependency>
<groupId>org.springframework</groupId>
<artifactId>spring-core</artifactId>
<version>3.2.8.RELEASE</version>
</dependency>
And maven will automatically take care of the dependencies for me.
A good way to test this out is to open a new maven project in eclipse and specify a dependency such as this, update the project, and then check in the Maven dependencies folder.
For fun, I experimented with this and it is indeed true, Maven will download any necessary dependencies when you update your project. After putting only the above dependency in my pom.xml file I got the following:
No need to download all those.
Maven will take care of all the artifact's dependencies for the specified dependency mentioned in pom file.
The common ways of including external non-maven jar in the classpath of your Maven project is to either use "addjar-maven-plugin" (which I have not been able to get to compile maven with) or "mvn install:install-file" and then adding a dependency section for the external JAR. This approach essentially installs client JAR in your repo and makes it available in classpath for maven. But is there a better way to do this (or) are the options mentioned above the only ones available? I just want the external JAR to be added to classpath for component scanning by Spring but do not want the JAR itself to be added to my repo as it is client's JAR? I hope this is a valid case (if not, kindly explain)
Thanks,
Paddy
You can create lib folder under your project's src folder and reference this folder as maven repository.
<repository>
<id>local</id>
<url>file://${basedir}/src/lib</url>
</repository>
Then you should add dependency to your jar in your pom.xml
<dependency>
<groupId>com.company</groupId>
<artifactId>dependency</artifactId>
<version>1.0</version>
</dependency>
After that your should rename jar file and place it on following path src/lib/com/company/dependency/1.0/dependency-1.0.jar . This path depends on how you want to reference your jar.
I am using Netbeans to build a Maven project, and have the JTidy java library as a dependency. It turns out JTidy doesnt exist in any maven repos, so I can't just add a "normal" depedency entry for it.
What is the best way of handling dependencies to libraries in Maven projects that arent available on repos?
I've currently tried adding it to my maven pom as such (after copying the jar to my projects /libs folder)
<dependency>
<groupId>org.w3c</groupId>
<artifactId>org.w3c.tidy</artifactId>
<version>9.3.8</version>
<scope>system</scope>
<systemPath>${basedir}/libs/jtidy-r938.jar</systemPath>
</dependency>
However it complains that it will be unresolvable by dependent projects.
First of all, it's under another groupId, that's why you didn't find it.
<dependency>
<groupId>net.sf.jtidy</groupId>
<artifactId>jtidy</artifactId>
<version>r938</version>
</dependency>
Jtidy
But to answer your question, one way of doing this is to manually install it in your local repo as described here.
The best way IMHO is to add it to a proxy like Nexus. That way other people can access it from there without having to install it locally. However, this means you have to set up a repository manager, which doesn't make much sense if you are the only developer on the project.
I'm converting an ANT project that uses CXF into a Maven one. The problem is that this projects depends on CXF v2.2.6 and when I go to here or even here, there is no jar to download. I don't know what to do. I have the JAR but I want to use the Maven features like dependency management.
Thanks for your help
If using Ant, it likely used the cxf-bundle jar that pretty much contains all of CXF. You can just add:
<dependency>
<groupId>org.apache.cxf</groupId>
<artifactId>cxf-bundle</artifactId>
<version>2.2.6</version>
</dependency>
to your new pom and it would get that jar along with all the dependencies that it would require.
It is available at Maven central. Use repo1.maven.org as your repository.
Either install it manually to your local repository or use a repository manager like Nexus.