When I use IntelIJ as IDE for spark applications, I can run the application in IDE while failed to run spark-submit after exporting this application with all the required dependencies.
Getting ERROR message as:
Exception in thread "main" java.lang.NoClassDefFoundError: javax/servlet/Servlet
at org.apache.spark.ui.WebUI.attachPage(WebUI.scala:80)
...
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
... 27 more
But when I have check the jar, javax/servlet/Servlet is just there! why?
I use jars with hadoop, spark and hive instead of maven pom. Will this matter? The jar file is huge in size (about 200M).
Finally, please forgive me for my pool English!
Related
The strange thing is that I don't use the missing class DispatchApplication and because I have no error when launching it using idea, so i do not know how to debug it.
I tried to clean idea cache, clean local maven repository, restart my PC
but my colleague can run with a jar, and each of them don't know why this happen and how solve this
D:\projectFolder\midel-data\data-integration\data-integration-service\target>java -jar data-integration-service-1.0.jar
Exception in thread "main" java.lang.ClassNotFoundException: com.lty.dispatch.DispatchApplication
at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
at org.springframework.boot.loader.LaunchedURLClassLoader.loadClass(LaunchedURLClassLoader.java:151)
at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:348)
at org.springframework.boot.loader.MainMethodRunner.run(MainMethodRunner.java:46)
at org.springframework.boot.loader.Launcher.launch(Launcher.java:108)
at org.springframework.boot.loader.Launcher.launch(Launcher.java:58)
at org.springframework.boot.loader.JarLauncher.main(JarLauncher.java:88)
This looks to be an issue with dependencies.
You might want to check your IDE settings if you have explicitly enabled any additional jars in the class path.
Also, check out your pom.xml/.gradle file for any difference in dependency versions.
If this class is part of some custom jar, make sure you include it while packaging your jar.
I am packaging a jar file with
gradlew desktop:dist
when running it with java 1.8 (openjdk)
java -jar desktop-1-0.jar
a ClassNotFoundException for the FreeTypeFontGenerator class is thrown. The whole Stacktrace is the following
java -jar desktop-1.0.jar
Picked up _JAVA_OPTIONS: -Xmx512M
Exception in thread "LWJGL Application" com.badlogic.gdx.utils.GdxRuntimeException:
java.lang.NoClassDefFoundError: com/badlogic/gdx/graphics/g2d/freetype/FreeTypeFontGenerator
at com.badlogic.gdx.backends.lwjgl.LwjglApplication$1.run(LwjglApplication.java:135)
Caused by: java.lang.NoClassDefFoundError: com/badlogic/gdx/graphics/g2d/freetype/FreeTypeFontGenerator
at com.mygdx.tools.FontLoader.loadFont(FontLoader.java:13)
at com.mygdx.stages.hud.ClientHud.(ClientHud.java:42)
at com.mygdx.stages.hud.StartmenuHud.joinButtonClicked(StartmenuHud.java:67)
at com.mygdx.stages.hud.StartmenuHud.access$100(StartmenuHud.java:16)
at com.mygdx.stages.hud.StartmenuHud$2.touchDown(StartmenuHud.java:47)
at com.badlogic.gdx.scenes.scene2d.InputListener.handle(InputListener.java:57)
at com.badlogic.gdx.scenes.scene2d.Actor.notify(Actor.java:188)
at com.badlogic.gdx.scenes.scene2d.Actor.fire(Actor.java:158)
at com.badlogic.gdx.scenes.scene2d.Stage.touchDown(Stage.java:281)
at com.badlogic.gdx.backends.lwjgl.LwjglInput.processEvents(LwjglInput.java:357)
at com.badlogic.gdx.backends.lwjgl.LwjglApplication.mainLoop(LwjglApplication.java:221)
at com.badlogic.gdx.backends.lwjgl.LwjglApplication$1.run(LwjglApplication.java:128)
Caused by: java.lang.ClassNotFoundException: com.badlogic.gdx.graphics.g2d.freetype.FreeTypeFontGenerator
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
... 12 more
I already included the necessary dependencies in the build.gradle file within the root directory of my project for FreeTypeFont as described in the libgdx wiki.
I also refreshed the project dependencies (right-click project -> Gradle -> Refresh Gradle Dependencies)
and rebuild the project.
The solution for me was to add
implementation "com.badlogicgames.gdx:gdx-freetype:$gdxVersion"
under the desktop section of my build.gradle file.
I tried this out since the import for the controler module looked a bit alike.
I have no idea why this is not documented, but after a refresh of all gradle dependencies (right-click the project -> Gradle -> Refresh Gradle Dependencies) and a rebuild of the project i was able to run the packaged jar without any problems.
At the moment I'm trying to run my first flink application. I already tested the java file (KMeans.java) in the IDE and it works perfectly but I can't handle to get this java file run as an jar in command line.
The build was successfully created with mvn clean package.
But if I run my jar file in command line flink run -c KMeans name.jar
this error message appears:
The program finished with the following exception:
org.apache.flink.client.program.ProgramInvocationException: The
program's entry point class 'KMeans' was not found in the jar file.
at
org.apache.flink.client.program.PackagedProgram.loadMainClass(PackagedProgram.java:617)
at
org.apache.flink.client.program.PackagedProgram.(PackagedProgram.java:199)
at
org.apache.flink.client.cli.CliFrontend.buildProgram(CliFrontend.java:856)
at org.apache.flink.client.cli.CliFrontend.run(CliFrontend.java:206)
at
org.apache.flink.client.cli.CliFrontend.parseParameters(CliFrontend.java:1044)
at
org.apache.flink.client.cli.CliFrontend.lambda$main$11(CliFrontend.java:1120)
at java.security.AccessController.doPrivileged(Native Method) at
javax.security.auth.Subject.doAs(Subject.java:422) at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1754)
at
org.apache.flink.runtime.security.HadoopSecurityContext.runSecured(HadoopSecurityContext.java:41)
at
org.apache.flink.client.cli.CliFrontend.main(CliFrontend.java:1120)
Caused by: java.lang.ClassNotFoundException: KMeans at
java.net.URLClassLoader.findClass(URLClassLoader.java:381) at
java.lang.ClassLoader.loadClass(ClassLoader.java:424) at
java.lang.ClassLoader.loadClass(ClassLoader.java:357) at
java.lang.Class.forName0(Native Method) at
java.lang.Class.forName(Class.java:348) at
org.apache.flink.client.program.PackagedProgram.loadMainClass(PackagedProgram.java:614)
... 10 more
So I looked up my generated target folder and there is a KMeans.class file in the classes folder. So I'm doing this wrong?
You need to specify the full class name, e.g., org.apache.flink.examples.java.clustering.KMeans.
Note that you only need to use the -c flag if the JAR file doesn't specify the class to run in its manifest.
I had this problem for last few weeks and whenever i run a java application the following exception :
java.lang.NoClassDefFoundError: com/sun/jersey/core/util/FeaturesAndProperties
Caused by: java.lang.ClassNotFoundException:
com.sun.jersey.core.util.FeaturesAndProperties
The problem was jerseyclient.jar, jersey.jar and javax.ws.rs.jar was there in JDK classpath and also there a dependency shich had these files so the that caused this exception. Hope it helps someone !!!
I am new to Hadoop and I have just setup Hadoop 1.2.1 on my Mac laptop (Mavericks). I then created a simple WordCount project in IntelliJ IDEA and was able to run the code on a dummy text file. I am having trouble with successfully creating a jar file which will replicate my execution through the IDE. I get the following error:
java -jar ./out/artifacts/WordCount_jar/WordCount.jar test.txt out [19:35:21]
Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/commons/logging/LogFactory
at org.apache.hadoop.conf.Configuration.<clinit>(Configuration.java:146)
at neu.cs.parallelprogramming.WordCount.main(WordCount.java:48)
Caused by: java.lang.ClassNotFoundException: org.apache.commons.logging.LogFactory
at java.net.URLClassLoader$1.run(URLClassLoader.java:372)
at java.net.URLClassLoader$1.run(URLClassLoader.java:361)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:360)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
... 2 more
FAIL: 1
Could anyone let me know what I am missing?
I guess you have to specify your class (which implements the Map/Reduce function).
E.g., $ java -jar ./WordCount.jar classWordCount input.txt output
or $ hadoop jar yourprogram.jar **yourclass** inputpath outputpath