I'm attempting to use the AWS Athena JDBC driver (AthenaJDBC42-2.0.2.jar)
described here.
If I just take the jar and add it to classpath I have no problems.
When I move to a gradle build and add a dependency to a local version of the jar, again no problems.
compile files('libs/AthenaJDBC42_2.0.2.jar')
But what I really want to do is just fetch the jar from a repository somewhere only I can't seem to find this jar available in any repository.
Does anyone know if there is something special about this particular jar and why it isn't available in a repo?
I tried this one and it worked
https://mvnrepository.com/artifact/com.syncron.amazonaws/simba-athena-jdbc-driver/2.0.2
<dependency>
<groupId>com.syncron.amazonaws</groupId>
<artifactId>simba-athena-jdbc-driver</artifactId>
<version>2.0.2</version>
</dependency>
Related
I am not able to find any hint about the issue that I am facing and hence posting my question here. Please apologize if it is something silly.
I have some working experience in selenium webdriver. However in my new project, I was asked to use an existing Selenium framework.
In the pom.xml, I am seeing the below dependency (I have edited xxx to avoid displaying the company name)
<dependencies>
<dependency>
<groupId>com.xxxqa.selenium</groupId>
<artifactId>selenium-central-framework</artifactId>
<version>2.0.18</version>
</dependency>
Question :
I couldn't understand from where that dependancy was taken from ?
Solution Tried :
I have tried searching in the maven repository still couldn't find
anything related to "selenium-central-framework"
I have checked the dependencies of selenium RC, but they seems to be different from the one that is present above.
I have researched whether any local jars were used and were linked
as dependancy. However I understand from maven repository, local
repositories will be linked with the tag "systemPath"
<dependency>
<groupId>ldapjdk</groupId>
<artifactId>ldapjdk</artifactId>
<scope>system</scope>
<version>1.0</version>
<systemPath>${basedir}\src\lib\ldapjdk.jar</systemPath>
</dependency>
</dependencies>
Note that the depedency used in my project doesn't have the tag "systemPath".
Is there anyway the external repositories could be added without mentioning systemPath ??
Please help, what could be that dependency ? How to find more information about it. Thanks a ton for your help.
I suggest you one of these approaches:
Jar file
If you have access to the jar file called : selenium-central-framework-2.0.18.jar, open a command line pointing to the folder of this jar and execute:
mvn install:install-file \
-Dfile=selenium-central-framework-2.0.18.jar \
-DgroupId=com.xxxqa.selenium \
-DartifactId=selenium-central-framework \
-Dversion=2.0.18 -Dpackaging=jar
Source code file
If you have the source code of : selenium-central-framework-2.0.18.jar, open a command line pointing to the folder of this source code and execute:
mvn clean install
After one of these approaches, selenium-central-framework-2.0.18.jar will be available in you local maven repository ($HOME/.m2) and your other java maven projects will be ready to use it as dependency:
<dependency>
<groupId>com.xxxqa.selenium</groupId>
<artifactId>selenium-central-framework</artifactId>
<version>2.0.18</version>
</dependency>
This is a workaround if you don't have a server to host an artifact repository management server like :
https://www.sonatype.com/nexus-repository-sonatype
https://www.jfrog.com/confluence/display/RTF/Maven+Repository
Explanation
Almost all free, public, safe, useful and cool java libraries are hosted in https://mvnrepository.com . So any person in the world can use it in their maven project with this piece in their pom.xml:
<dependency>
<groupId>org.seleniumhq.selenium</groupId>
<artifactId>selenium-java</artifactId>
<version>3.141.59</version>
</dependency>
But in some cases, publish your source code to https://mvnrepository.com is not an option:
Oracle Driver. This driver (.jar) can only be downloaded from the official Oracle page. Private source code like IBM, Microsfot,etc
Source code of your company that should not be public.
Some super cool library that is only in github but not in the maven central repository.
So in this cases, the best and scalable solution is host and configure some artifact repository management server like :
https://www.sonatype.com/nexus-repository-sonatype
https://www.jfrog.com/confluence/display/RTF/Maven+Repository
This platforms are an emulation of https://mvnrepository.com . So with a minimal configuration:
You can host your special or private jars
Your maven projects could use this jars with the standrad xml dependency in pom.xml.
If you don't have a server to implement one of these platforms, the previous approaches could help you and get the same results
I'm working on a recommender system using Apache Flink. The implementation is running when I test it in IntelliJ, but I would like now to go on a cluster. I also built a jar file and tested it locally to see if all was working but I encountered a problem.
java.lang.NoClassDefFoundError: org/apache/flink/ml/common/FlinkMLTools$
As we can see, the class FlinkMLTools used in my code isn't found during the running of the jar.
I built this jar with Maven 3.3.3 with mvn clean install and I'm using the version 0.9.0 of Flink.
First Trail
The fact is that my global project contains other projects (and this recommender is one of the sub-project). In that way, I have to launch the mvn clean install in the folder of the right project, otherwise Maven always builds a jar of an other project (and I don't understand why). So I'm wondering if there could be a way to say explicitly to maven to build one specific project of the global project. Indeed, perhaps the path to FlinkMLTools is contained in a link present in the pom.xml file of the global project.
Any other ideas?
The problem is that Flink's binary distribution does not contain the libraries (flink-ml, gelly, etc.). This means that you either have to ship the library jar files with your job jar or that you have to copy them manually to your cluster. I strongly recommend the first option.
Building a fat-jar to include library jars
The easiest way to build a fat jar which does not contain unnecessary jars is to use Flink's quickstart archetype to set up the project's pom.
mvn archetype:generate -DarchetypeGroupId=org.apache.flink \
-DarchetypeArtifactId=flink-quickstart-scala -DarchetypeVersion=0.9.0
will create the structure for a Flink project using the Scala API. The generated pom file will have the following dependencies.
<dependencies>
<dependency>
<groupId>org.apache.flink</groupId>
<artifactId>flink-scala</artifactId>
<version>0.9.0</version>
</dependency>
<dependency>
<groupId>org.apache.flink</groupId>
<artifactId>flink-streaming-scala</artifactId>
<version>0.9.0</version>
</dependency>
<dependency>
<groupId>org.apache.flink</groupId>
<artifactId>flink-clients</artifactId>
<version>0.9.0</version>
</dependency>
</dependencies>
You can remove flink-streaming-scala and instead you insert the following dependency tag in order to include Flink's machine learning library.
<dependency>
<groupId>org.apache.flink</groupId>
<artifactId>flink-ml</artifactId>
<version>0.9.0</version>
</dependency>
When you know build the job jar with mvn package, the generated jar should contain the flink-ml jar and all of its transitive dependencies.
Copying the library jars manually to the cluster
Flink includes all jars which are located in the <FLINK_ROOT_DIR>/lib folder in the classpath of the executed jobs. Thus, in order to use Flink's machine learning library you have to put the flink-ml jar and all needed transitive dependencies into the /lib folder. This is rather tricky, since you have to figure out which transitive dependencies are actually needed by your algorithm and, consequently, you will often end up copying all transitive dependencies.
How to build a specific sub-module with maven
In order to build a specific sub-module X from your parent project you can use the following command:
mvn clean package -pl X -am
-pl allows you to specify which sub-modules you want to build and -am tells maven to also build other required sub-modules. It is also described here.
In cluster mode, Flink does not put all library JAR files into the classpath of its workers. When executing the program locally in IntelliJ all required dependencies are in the classpath, but not when executing on a cluster.
You have two options:
copy the FlinkML Jar file into the lib folder of all Flink TaskManager
Build a fat Jar file for you application that includes the FLinkML dependencies.
See the Cluster Execution Documentation for details.
The XADisk library deployed on Maven Central packaged as 'rar' instead of 'jar'. But i just need the jar (and possibly source) for the project i'm working on. I was wondering what the best (maven style) way is to deal with this dependency.
The jar files are available on Central but not specified in the pom thus type="jar" doesnt work
the pom is here: https://repo1.maven.org/maven2/net/java/xadisk/xadisk/1.2.2/xadisk-1.2.2.pom
and the jars can be found here: https://repo1.maven.org/maven2/net/java/xadisk/xadisk/1.2.2/xadisk-1.2.2.pom
I can't reproduce your issue, maybe they changed something in the repo?
If I add the following dependency to my project:
<dependency>
<groupId>net.java.xadisk</groupId>
<artifactId>xadisk</artifactId>
<version>1.2.2</version>
</dependency>
then the JAR file gets packaged into my project.
By the way, if no "type" is specified (for the dependency), maven uses JAR as default.
I have a maven project B which is packaged as a war B.war and has a 'local' dependency A.jar. The pom for building A.jar has a dependency on restFB and it resolves properly while compiling.
<dependency>
<groupId>com.restfb</groupId>
<artifactId>restfb</artifactId>
<version>${com.restfb-version}</version>
</dependency>
However when I package B.war, restFB's jar is not present in the WEB-INF/lib directory of B.war and execution throws NoClassDefFoundError. What is also baffling is that I find this happening only when I build it on an AWS Amazon Linux and not while building on Ubuntu. There are similar questions in SO which suggest adding
<packaging>war</packaging>
which I already have but doesn't seem to solve the problem. Any ideas how to solve this?
It should be simple to understand, that you might have invoke locally under Ubuntu
mvn install
for A. Therefore it is in the local .m2 repository.
Did AWS Amazon Linux also have this artifact in its repository? If not, copy it there and try to package again.
Also you explicitly include and exclude certain artifacts within the build configuration for maven-war-plugin
I'm converting an ANT project that uses CXF into a Maven one. The problem is that this projects depends on CXF v2.2.6 and when I go to here or even here, there is no jar to download. I don't know what to do. I have the JAR but I want to use the Maven features like dependency management.
Thanks for your help
If using Ant, it likely used the cxf-bundle jar that pretty much contains all of CXF. You can just add:
<dependency>
<groupId>org.apache.cxf</groupId>
<artifactId>cxf-bundle</artifactId>
<version>2.2.6</version>
</dependency>
to your new pom and it would get that jar along with all the dependencies that it would require.
It is available at Maven central. Use repo1.maven.org as your repository.
Either install it manually to your local repository or use a repository manager like Nexus.