I'm currently building apps for Apache Spark. Spark provides during runtime a lot of dependencies, which I normally need if I test/run the apps locally in the IDE (IntelliJ).
Is there any possibility to have different set of dependendencies related if I use the 'package' or the usual compile/run target in IntelliJ ?
For instance, this is a needed dependency to Hadoop
<dependency>
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-common</artifactId>
<version>2.6.0</version>
<scope>provided</scope>
</dependency>
But the scope 'provided' does not work when I run it locally in the IDE.
If you want IntelliJ to use its own build process rather than Maven's, it's probably better to tell add a (global) library to your project dependencies in the IDE.
It definitely won't be providing these Spark JARs by default, which is what you're telling Maven here.
Related
I have a given eclipse maven project which builds to a jar. The pom has one major dependency of BiRT 4.8.0-202010080643 Runtime.
<dependency>
<groupId>com.customer.birt.runtime</groupId>
<artifactId>org.eclipse.birt.runtime</artifactId>
<version>4.8.0-202010080643</version>
</dependency>
So they pushed the artifact into their own nexus; thats why com.customer.birt.runtime.
I really don't know how the guy did that and which tools he used. Currently I want to update to BiRT 4.9. Replacing the above with the only available:
<dependency>
<groupId>org.eclipse.birt</groupId>
<artifactId>birt-runtime</artifactId>
<version>4.9.0</version>
<type>pom</type>
</dependency
does not go well. Both are totally different constellations from the same big project. How can I make use of the above maven dependency of 4.9 in my simple birt project? I'm building only a service for a desktop application that is hosted and run within an RCP application. I started to list the individual maven deps so that the java compiles which I succeeded to but I still have few unit tests that execute and render ReportEngine and fail because of missing Deps at runtime. This is because the ReportEngine is loading APIs at runtime..
I started to post here once I noticed that I will be declaring the separate deps in pom.xml blindly which is (even if the Unittests pass) very unreliable..
Thank you so much!
M.Abdu
My solution was currently as I put in the comments or yet simpler. I just uploaded manually the birt-runtime jar into nexus using my account within the customer and then put in my pom the exact same unique coordinates groupid:artifactid:version. Plus some other dependencies depending of what my unit tests are asking at runtime, e.g. eclipse.platform, emf.core, w3c, batik.css etc.
I am talking about executing the build using mvn clean verify and resulting a jar file
The jar you get from here
https://search.maven.org/remotecontent?filepath=org/eclipse/birt/birt-runtime/4.9.0/birt-runtime-4.9.0.zip
pom in my case:
<dependency>
<groupId>org.eclipse.birt</groupId>
<artifactId>runtime</artifactId>
<version>4.9.0-20220502</version>
</dependency>
I have a project where several dependency versions need to be chosen at deployment time - i.e. specified in the classpath.
The provided scope prevents the dependency being packaged but the project fails when I try to run from within IntelliJ IDEA
e.g.
<dependency>
<groupId>org.apache.activemq</groupId>
<artifactId>activemq-core</artifactId>
<version>5.3.1</version>
<scope>provided</scope>
</dependency>
produces
{stacktrace ...}
Caused by: java.lang.ClassNotFoundException: javax.jms.ConnectionFactory
If I remove the scope the project runs fine but, of course, includes the jar.
If you mark a jar as <provided>, the classes need to be provided by the container that runs the surrounding war/ear.
When you run your project from within IntelliJ, it is probably deployed on some kind of container. Make sure this container provides you <provided> dependencies.
Final approach was to create two maven profiles, one for running locally and one for packaging. The local profile used compile scope while the package profile used provided.
I'm working on a recommender system using Apache Flink. The implementation is running when I test it in IntelliJ, but I would like now to go on a cluster. I also built a jar file and tested it locally to see if all was working but I encountered a problem.
java.lang.NoClassDefFoundError: org/apache/flink/ml/common/FlinkMLTools$
As we can see, the class FlinkMLTools used in my code isn't found during the running of the jar.
I built this jar with Maven 3.3.3 with mvn clean install and I'm using the version 0.9.0 of Flink.
First Trail
The fact is that my global project contains other projects (and this recommender is one of the sub-project). In that way, I have to launch the mvn clean install in the folder of the right project, otherwise Maven always builds a jar of an other project (and I don't understand why). So I'm wondering if there could be a way to say explicitly to maven to build one specific project of the global project. Indeed, perhaps the path to FlinkMLTools is contained in a link present in the pom.xml file of the global project.
Any other ideas?
The problem is that Flink's binary distribution does not contain the libraries (flink-ml, gelly, etc.). This means that you either have to ship the library jar files with your job jar or that you have to copy them manually to your cluster. I strongly recommend the first option.
Building a fat-jar to include library jars
The easiest way to build a fat jar which does not contain unnecessary jars is to use Flink's quickstart archetype to set up the project's pom.
mvn archetype:generate -DarchetypeGroupId=org.apache.flink \
-DarchetypeArtifactId=flink-quickstart-scala -DarchetypeVersion=0.9.0
will create the structure for a Flink project using the Scala API. The generated pom file will have the following dependencies.
<dependencies>
<dependency>
<groupId>org.apache.flink</groupId>
<artifactId>flink-scala</artifactId>
<version>0.9.0</version>
</dependency>
<dependency>
<groupId>org.apache.flink</groupId>
<artifactId>flink-streaming-scala</artifactId>
<version>0.9.0</version>
</dependency>
<dependency>
<groupId>org.apache.flink</groupId>
<artifactId>flink-clients</artifactId>
<version>0.9.0</version>
</dependency>
</dependencies>
You can remove flink-streaming-scala and instead you insert the following dependency tag in order to include Flink's machine learning library.
<dependency>
<groupId>org.apache.flink</groupId>
<artifactId>flink-ml</artifactId>
<version>0.9.0</version>
</dependency>
When you know build the job jar with mvn package, the generated jar should contain the flink-ml jar and all of its transitive dependencies.
Copying the library jars manually to the cluster
Flink includes all jars which are located in the <FLINK_ROOT_DIR>/lib folder in the classpath of the executed jobs. Thus, in order to use Flink's machine learning library you have to put the flink-ml jar and all needed transitive dependencies into the /lib folder. This is rather tricky, since you have to figure out which transitive dependencies are actually needed by your algorithm and, consequently, you will often end up copying all transitive dependencies.
How to build a specific sub-module with maven
In order to build a specific sub-module X from your parent project you can use the following command:
mvn clean package -pl X -am
-pl allows you to specify which sub-modules you want to build and -am tells maven to also build other required sub-modules. It is also described here.
In cluster mode, Flink does not put all library JAR files into the classpath of its workers. When executing the program locally in IntelliJ all required dependencies are in the classpath, but not when executing on a cluster.
You have two options:
copy the FlinkML Jar file into the lib folder of all Flink TaskManager
Build a fat Jar file for you application that includes the FLinkML dependencies.
See the Cluster Execution Documentation for details.
I have a Maven project in which I use a javaxf WebEngine. I first included javafx by installing e(fx)clipse, and I was able to use it normally.
Now I want to compile my program to a big .jar file with all dependencies included. I first used a <scope>system</scope> and linked to the jfxrt.jar in my JDK (1.7.0_45). I compile my program using mvm package and it works well for the build part.
Then, I installed the jfxrt.jar in maven thanks to this SO answer so I have a javafx-2.2.45.jar with maven.
However, I'm stuck with this error when I run my program and when it comes to instantiate that particular WebEngine:
INFO: com.sun.javafx.css.StyleManager loadStylesheetUnPrivileged Could not find stylesheet: jar:file:/target/project-name-0.1.one-jar.jar!/lib/javafx-2.2.45.jar!/com/sun/javafx/scene/control/skin/caspian/caspian.css
SEVERE: javafx.scene.control.Control impl_processCSS The -fx-skin property has not been defined in CSS for ScrollBarThemeImpl$ScrollBarWidget#5919e0a8[styleClass=scroll-bar]
java.lang.NullPointerException
at com.sun.webpane.sg.theme.ScrollBarThemeImpl.initializeThickness(ScrollBarThemeImpl.java:341)
at com.sun.webpane.sg.theme.ScrollBarThemeImpl.access$100(ScrollBarThemeImpl.java:27)
at com.sun.webpane.sg.theme.ScrollBarThemeImpl$ScrollBarWidget.impl_updatePG(ScrollBarThemeImpl.java:50)
at javafx.scene.Node.impl_syncPGNode(Node.java:425)
at javafx.scene.Scene$ScenePulseListener.syncAll(Scene.java:2106)
at javafx.scene.Scene$ScenePulseListener.syncAll(Scene.java:2115)
at javafx.scene.Scene$ScenePulseListener.syncAll(Scene.java:2115)
at javafx.scene.Scene$ScenePulseListener.synchronizeSceneNodes(Scene.java:2082)
at javafx.scene.Scene$ScenePulseListener.pulse(Scene.java:2193)
at com.sun.javafx.tk.Toolkit.firePulse(Toolkit.java:363)
at com.sun.javafx.tk.quantum.QuantumToolkit.pulse(QuantumToolkit.java:463)
at com.sun.javafx.tk.quantum.QuantumToolkit$9.run(QuantumToolkit.java:332)
at com.sun.glass.ui.InvokeLaterDispatcher$Future.run(InvokeLaterDispatcher.java:76)
I opened this jar and was able to find the caspian.css file where it's supposed to be.
What should I do to be able to use my WebEngine in my "big compiled jar"? I don't care if the solution is a quick, dirty fix such as copy/paste of this css file somewhere else (I already tried that but I might have missed something..)
It's not a good idea to package fx into your jar. Have you tried using http://www.zenjava.com/2013/07/01/javafx-maven-plugin-2-0-released/
You might need to declare your dependency like for not to include JavaFX into the uber jar. You could also exclude javafx packages from when creating this jar, but I don't think it's recommended as in theory resources might not be placed under the javafx package.
<dependency>
<groupId>com.oracle</groupId>
<artifactId>javafx</artifactId>
<version>2.2.3</version>
<scope>provided</scope>
</dependency>
or
<dependency>
<groupId>com.oracle</groupId>
<artifactId>javafx</artifactId>
<version>2.2.3</version>
<optional>true</optional>
</dependency>
Update: the is also a javafx maven plugin as #tomsontom mentioned.
I have a Maven project that just displays a graph on the xy axis. I want to change that graph to a Javafx 2.0 linechart to display the same data. I tried using the FEST-javafx-maven plugin, but I still cannot compile the code; the compiler cannot find all of the javafx.xxx imports.
Any help would be appreciated.
Update (Oct 6 2015)
Modern JavaFX versions (shipped with Oracle Java 8+) do not require any additional class path to use JavaFX. The JavaFX runtime is on the default classpath Java uses for compilation and execution. This means that a plain maven pom.xml file, with no additional dependencies, will build a JavaFX application.
However, if you wish to use additional packaging features for your application, such as the ability to deploy it as a self-contained application, then I advise using the (third party) JavaFX Maven plugin.
Previous Answer
The following information in this answer is now mostly old and outdated.
The link to the fest maven plugin I found (http://fest.easytesting.org/javafx/maven/) is to a tool for building JavaFX 1.x script projects, which is a completely different and incompatible beast to JavaFX 2.0 - I'm not sure if there is an updated version of the fest maven plugin which supports JavaFX 2.0.
There is currently no official support for Maven from Oracle, nor a version of JavaFX 2.0 in a publicly hosted Maven repository.
However, I have successfully built JavaFX 2.0 projects using maven in the past by using a system scoped dependency on the jfxrt.jar and (optionally) invoking the JavaFX ant tasks from maven.
If you are embedding your graph in an existing Swing application via a JFXPanel, then you don't need to use the JavaFX ant tasks. Add jfxrt.jar from the JavaFX runtime as a system dependency OR manually install it into your maven repository to use a non-system scoped dependency.
An example of the command to manually install the required JavaFX 2.0 runtime jar is:
mvn install:install-file -Dfile="C:\Program Files\Oracle\JavaFX 2.1.0 SDK\rt\lib\jfxrt.jar" -DgroupId=com.oracle.javafx -DartifactId=javafx -Dversion=2.1 -Dpackaging=jar
After running the above command, add the dependency to the jfxrt.jar file to your maven pom and your project compilation should resolve all JavaFX API references:
<dependency>
<groupId>com.oracle.javafx</groupId>
<artifactId>javafx</artifactId>
<version>2.1</version>
</dependency>
If you extend the JavaFX Application class and you want your application packaged for deployment via webstart, browser embedding or as a JavaFX installation aware clickable jar, then you should adapt your pom.xml file to execute the relevant JavaFX 2.0 ant tasks - http://code.google.com/p/willow-browser/source/browse/pom.xml demonstrates such an adaption.
These threads discuss JavaFX 2.0 maven support and provide further background info and samples:
https://forums.oracle.com/forums/thread.jspa?messageID=9970002
http://mail.openjdk.java.net/pipermail/openjfx-dev/2011-December/000076.html
I had the same problem and here is my solution:
If using Java 7u7 (javafx is integrated into jdk/jre):
<dependency>
<groupId>javafx</groupId>
<artifactId>javafx</artifactId>
<version>2.1</version>
<scope>system</scope>
<systemPath>${java.home}/lib/jfxrt.jar</systemPath>
</dependency>
For previous versions of java:
<dependency>
<groupId>javafx</groupId>
<artifactId>javafx</artifactId>
<version>2.1</version>
<scope>system</scope>
<systemPath>${env.JAVAFX_HOME}\rt\lib\jfxrt.jar</systemPath>
</dependency>
And you have to set system variable JAVAFX_HOME to home dir of JavaFx.
You might try the JavaFX Maven plugin. This takes care of adding javafx to the classpath as well as building JavaFX apps and creating Windows/Mac/Linux executables, as well as double-clickable JARs and JNLP files.
Using JDK 1.7 you have to perform the following mvn goal.
mvn com.zenjava:javafx-maven-plugin:2.0:fix-classpath
This command will change the classpath of your JRE and copy the jfxrt.jar to the JAVA_HOME\lib\ext directory.
Take an additional look here for further information:
JavaFX Maven Plugin
Within your pom-file you have to add the following dependency configuration.
<dependency>
<groupId>javafx</groupId>
<artifactId>jfxrt</artifactId>
<version>2.2</version>
<!-- <version>${javafx.min.version}</version> -->
<scope>system</scope>
<systemPath>${java.home}/lib/ext/jfxrt.jar</systemPath>
</dependency>