I have a Maven project that's building fine, and I'm attempting to add a reference to twitter4j.
So I add this to the pom:
<dependency>
<groupId>org.twitter4j</groupId>
<artifactId>twitter4j-core</artifactId>
<version>[2.2,)</version>
<type>jar</type>
<optional>false</optional>
<scope>compile</scope>
</dependency>
When I build (mvn clean install) I get:
[ERROR] Error building bundle net.stevex.tweetfetcher:tweetfetcher-bundle:bundle:1.0-SNAPSHOT : Unresolved references to [twitter4j] by class(es) on the Bundle-Classpath[Jar:dot]: [net/stevex/tweetfetcher/impl/TweetFetcherImpl.class]
This makes sense .. the twitter4j package isn't embedded in the bundle. So I add an Embed-Dependency to the maven-bundle-plugin's instructions:
<Embed-Dependency>*;scope=compile|runtime</Embed-Dependency>
And now when I build, I get:
[ERROR] Error building bundle net.stevex.tweetfetcher:tweetfetcher-bundle:bundle:1.0-SNAPSHOT : Unresolved references to [dalvik.system, javax.crypto, javax.crypto.spec, javax.management, javax.management.openmbean, org.apache.commons.logging, org.apache.log4j, org.slf4j.impl, twitter4j.internal.http.alternative] by class(es) on the Bundle-Classpath[Jar:dot, Jar:twitter4j-core-2.2.3.jar]: [twitter4j/internal/logging/CommonsLoggingLogger.class, twitter4j/internal/logging/Log4JLoggerFactory.class, twitter4j/auth/OAuthToken.class, twitter4j/internal/http/HttpClientFactory.class, twitter4j/auth/OAuthAuthorization.class, twitter4j/internal/logging/Log4JLogger.class, twitter4j/conf/ConfigurationBase.class, twitter4j/TwitterAPIMonitor.class, twitter4j/internal/logging/CommonsLoggingLoggerFactory.class, twitter4j/management/APIStatisticsOpenMBean.class, twitter4j/internal/logging/Logger.class]
I don't understand why the twitter4j classes are missing, and I don't understand the references to dalvik.system, javax.crypto, etc. What's wrong here?
The problem is that the twitter4j project requires all of the packages listed in the error list. When you use it does not include all transitive dependencies. There is an Embed Transitive instruction for the maven-bundle-plugin that will embed all transitive dependencies
<Embed-Dependency>*;scope=compile|runtime</Embed-Dependency>
<Embed-Transitive>true</Embed-Transitive>
However, this kind of defeats the purpose of OSGi. This leaves you with two options that I know of:
Search for the packages in the second in the SpringSource EBR. You simply enter the package name in the search box, and the results will include the xml for your POM file. The SpringSource bundles will include references to other bundles in the EBR, eliminating the problem of transitive dependencies.
Use the bundle-all goal of the maven-bundle-plugin. This goal will run the maven bundle plugin for every dependency of your project, and place the resulting bundles into your target directory. You could then install these bundles into your local repository.
I would recommend using option 1 for as many of the bundles as you can find, then defaulting to option 2 when SpringSource doesn't have them.
Related
I am migrating my project from ant build to maven. For the third party dependencies, we have been using a custom maven repository (url http://SOME_IP/libs).I have defined this repository in my pom file and included these dependency like this :
<dependency>
<groupId>lib</groupId>
<artifactId>struts-core</artifactId>
<version>1.3.8</version>
<scope>runtime</scope>
</dependency>
Thus all dependencies are getting downloaded in .m2/repository/lib but when I run mvn compile it gives me compilation error saying that those package doesn't exist.
When you set scope to runtime, the dependencies are not available for compilation. You need to set scope to compile (or leave it out, which is the same).
I'm having trouble correctly importing a library into a project that I'm running. I have added the library as a dependency in the .pom, refreshed the pom, run mvn clean install, and I have set auto-import up so that the project gets updated correctly, but the project does not get added as an External Library, and I can't use it in my project. I get no errors. What am I doing wrong?
Here is the relevant part of my pom
..properties
<crowd.version>2.5.0</crowd.version>
.. end properties
<dependencyManagement>
<dependencies>
<dependency>
<groupId>com.atlassian.crowd</groupId>
<artifactId>crowd-integration-springsecurity</artifactId>
<version>${crowd.version}</version>
<scope>runtime</scope>
</dependency>
</dependencies>
</dependencyManagement>
Here is the question I was following to debug my error:
Import Maven dependencies in IntelliJ IDEA
I think you missed the point of dependency management; read more in official docs. This is a feature that you can centralize common dependency information that is then shared been different projects. All by itself, this definioition will not import the dependency.
What you probably want is just a plain dependency: drop the dependencyManagement tags, and move you dependency into the correct block in the pom.
I'm working on a recommender system using Apache Flink. The implementation is running when I test it in IntelliJ, but I would like now to go on a cluster. I also built a jar file and tested it locally to see if all was working but I encountered a problem.
java.lang.NoClassDefFoundError: org/apache/flink/ml/common/FlinkMLTools$
As we can see, the class FlinkMLTools used in my code isn't found during the running of the jar.
I built this jar with Maven 3.3.3 with mvn clean install and I'm using the version 0.9.0 of Flink.
First Trail
The fact is that my global project contains other projects (and this recommender is one of the sub-project). In that way, I have to launch the mvn clean install in the folder of the right project, otherwise Maven always builds a jar of an other project (and I don't understand why). So I'm wondering if there could be a way to say explicitly to maven to build one specific project of the global project. Indeed, perhaps the path to FlinkMLTools is contained in a link present in the pom.xml file of the global project.
Any other ideas?
The problem is that Flink's binary distribution does not contain the libraries (flink-ml, gelly, etc.). This means that you either have to ship the library jar files with your job jar or that you have to copy them manually to your cluster. I strongly recommend the first option.
Building a fat-jar to include library jars
The easiest way to build a fat jar which does not contain unnecessary jars is to use Flink's quickstart archetype to set up the project's pom.
mvn archetype:generate -DarchetypeGroupId=org.apache.flink \
-DarchetypeArtifactId=flink-quickstart-scala -DarchetypeVersion=0.9.0
will create the structure for a Flink project using the Scala API. The generated pom file will have the following dependencies.
<dependencies>
<dependency>
<groupId>org.apache.flink</groupId>
<artifactId>flink-scala</artifactId>
<version>0.9.0</version>
</dependency>
<dependency>
<groupId>org.apache.flink</groupId>
<artifactId>flink-streaming-scala</artifactId>
<version>0.9.0</version>
</dependency>
<dependency>
<groupId>org.apache.flink</groupId>
<artifactId>flink-clients</artifactId>
<version>0.9.0</version>
</dependency>
</dependencies>
You can remove flink-streaming-scala and instead you insert the following dependency tag in order to include Flink's machine learning library.
<dependency>
<groupId>org.apache.flink</groupId>
<artifactId>flink-ml</artifactId>
<version>0.9.0</version>
</dependency>
When you know build the job jar with mvn package, the generated jar should contain the flink-ml jar and all of its transitive dependencies.
Copying the library jars manually to the cluster
Flink includes all jars which are located in the <FLINK_ROOT_DIR>/lib folder in the classpath of the executed jobs. Thus, in order to use Flink's machine learning library you have to put the flink-ml jar and all needed transitive dependencies into the /lib folder. This is rather tricky, since you have to figure out which transitive dependencies are actually needed by your algorithm and, consequently, you will often end up copying all transitive dependencies.
How to build a specific sub-module with maven
In order to build a specific sub-module X from your parent project you can use the following command:
mvn clean package -pl X -am
-pl allows you to specify which sub-modules you want to build and -am tells maven to also build other required sub-modules. It is also described here.
In cluster mode, Flink does not put all library JAR files into the classpath of its workers. When executing the program locally in IntelliJ all required dependencies are in the classpath, but not when executing on a cluster.
You have two options:
copy the FlinkML Jar file into the lib folder of all Flink TaskManager
Build a fat Jar file for you application that includes the FLinkML dependencies.
See the Cluster Execution Documentation for details.
I'm having a slight problem with incorporating the appropriate maven dependencies into my project for sling models.
When I deploy my bundle, I get the following import that can't be resolved:
org.apache.sling.models.annotations,version=[1.1,2) -- Cannot be resolved
I believe I have included this with the following dependencies:
<dependency>
<groupId>org.apache.sling</groupId>
<artifactId>org.apache.sling.models.api</artifactId>
</dependency>
<dependency>
<groupId>org.apache.sling</groupId>
<artifactId>org.apache.sling.models.impl</artifactId>
</dependency>
I tried using:
<Embed-Transitive>true</Embed-Transitive>
<Import-Package>*</Import-Package>
in my bundle compile instructions, but this has just resulted in a ton of other dependencies not being resolved.
Surely I've gone down the garden path here somewhere. Any help would be greatly appreciated.
The org.apache.sling.models.api V1.0.2 bundle does export the following packages:
javax.inject,version=0.0.0
org.apache.sling.models.annotations,version=1.1.0
org.apache.sling.models.annotations.injectorspecific,version=1.0.0
org.apache.sling.models.spi,version=1.0.0
org.apache.sling.models.spi.injectorspecific,version=1.0.0
So if that bundle is active in your Sling instance, the org.apache.sling.models.annotations,version=[1.1,2) import should resolve.
Note that adding bundles to your maven dependencies might not be sufficient to install them in the running instance, what matters is whether the models.api bundle is active as seen from /system/console/bundles
Using Embed-Transitive is almost always a terrible idea. It traverses the entire transitive dependency hierarchy in Maven and pulls each one of those JARs into your own JAR. As a result you inherit all the package dependencies of all that crap you have dragged in.
When you have a bundle such as yours that requires an import -- in this case org.apache.sling.models.annotations -- the best solution is to find another bundle already available that exports the same package.
I have got two Maven projects. One of them has to be dependent on the other. I use IntelliJ and I have tried to right click on project1 > Open Module Settings, and in the dependencies tab I clicked on the + symbol to add a directory or jar dependency. So far so good, when I try to import packages from the dependency it autocompletes it for me, however the compilation throws errors, saying that there are no such packages. What am I doing wrong ?
There is no notion of project in Maven.
You have a Maven project B. You chose its groupId (com.mycompany, for example), its artifactId (B, for example), and its version (1.0-SNAPSHOT, for example). You run mvn install on this project. This generates a B-1.0-SNAPSHOT.jar file and stores it in your local Maven repository, with its pom.
Now you want to use B-1.0-SNAPSHOT.jar in another Maven project A. For A, B is a library, just like any other library you use (log4J, Spring, Hibernate, Guava, whatever). So you add a dependency to it in the pom of A, just like you do for any other library:
<dependencies>
<dependency>
<groupId>com.mycompany</groupId>
<artifactId>B</artifactId>
<version>1.0-SNAPSHOT</version>
</dependency>
<!-- other dependencies: log4J, Spring, Hibernate, Guava, whatever -->
</dependencies>
Read the awful documentation for more details.