I have main class XML.scala, Project XMLHDFS
I am trying to run spark submit job with command
spark-submit --class com.abc.Import.XMLHDFS.XML /some-path/XMLHDFS.jar
it is giving me class not found exception. Please suggest where I am wrong.
java.lang.ClassNotFoundException: com.abc.Import.XMLHDFS.XML
at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:278)
at org.apache.spark.util.Utils$.classForName(Utils.scala:172)
at org.apache.spark.util.Utils$.classForName(Utils.scala:172)
......................
Related
I installed giraph-1.0.0 on my 2-node cluster, built with hadoop version 2.0.4-alpha where as i have hadoop 2.7.1 installed on my cluster. When I tried to run the giraph example for Simple Shortest Path Vertex, I got the following NoClassDefFoundError.
hadoop jar $GIRAPH_HOME/giraph-examples/target/giraph-examples-1.0.0-for-hadoop-2.0.4-alpha-jar-with-dependencies.jar org.apache.giraph.GiraphRunner org.apache.giraph.examples.SimpleShortestPathsVertex -vif org.apache.giraph.io.formats.JsonLongDoubleFloatDoubleVertexInputFormat -vip /user/hduser/input/tinygraph.txt -of org.apache.giraph.io.formats.IdWithValueTextOutputFormat -op /user/hduser/output/shortestpaths -w 1
The error is
16/08/05 14:26:09 INFO utils.ConfigurationUtils: No edge input format specified. Ensure your InputFormat does not require one.
16/08/05 14:26:09 WARN job.GiraphConfigurationValidator: Output format vertex index type is not known
16/08/05 14:26:09 WARN job.GiraphConfigurationValidator: Output format vertex value type is not known
16/08/05 14:26:09 WARN job.GiraphConfigurationValidator: Output format edge value type is not known
Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/hadoop/yarn/client/YarnClientImpl
at java.lang.ClassLoader.defineClass1(Native Method)
at java.lang.ClassLoader.defineClass(ClassLoader.java:803)
at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
at java.net.URLClassLoader.defineClass(URLClassLoader.java:449)
at java.net.URLClassLoader.access$100(URLClassLoader.java:71)
at java.net.URLClassLoader$1.run(URLClassLoader.java:361)
at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
at org.apache.giraph.GiraphRunner.run(GiraphRunner.java:83)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
at org.apache.giraph.GiraphRunner.main(GiraphRunner.java:126)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.yarn.client.YarnClientImpl
at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
... 21 more
The command I used for building giraph is this.
mvn -Phadoop_yarn -DskipTests -Dhadoop.version=2.0.4-alpha clean package
I am not sure if I need to rebuild apache giraph or it is something else which is the problem. Thanks in advance.
Maybe it is because of my skills in handling the terminal, but I don't understand why I get an exception here.
I have extracted the folders. And I think the paths are right.
db#computer:~/Desktop/elki-0.7.0~20150828/elki$ java -cp elki-0.7.0~20150828.jar de.lmu.ifi.dbs.elki.application.GeneratorXMLSpec -app.out 0.txt -bymodel.spec 1.xml
Exception in thread "main" java.lang.NoClassDefFoundError: gnu/trove/list/TIntList
at java.lang.Class.getDeclaredMethods0(Native Method)
at java.lang.Class.privateGetDeclaredMethods(Class.java:2615)
at java.lang.Class.getMethod0(Class.java:2856)
at java.lang.Class.getMethod(Class.java:1668)
at sun.launcher.LauncherHelper.getMainMethod(LauncherHelper.java:494)
at sun.launcher.LauncherHelper.checkAndLoadMain(LauncherHelper.java:486)
Caused by: java.lang.ClassNotFoundException: gnu.trove.list.TIntList
at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
... 6 more
db#computer:~/Desktop/elki-0.7.0~20150828/elki$
Your Java classpath is incomplete.
Specifically, it does not include the Trove library required.
I'm building a customized version of Firefox Fennec, and recently added an UI that uses android's RecyclerView. So I want to import android-support-v7-recyclerview.jar to my moz.build file but don't know how.
I have tried to add android-support-v7-recyclerview.jar to gbjar.extra_jars, but it gives me exceptions below.
Exception in thread "main" java.lang.NoClassDefFoundError: android/support/v7/widget/RecyclerView$LayoutManager
0:46.88 at java.lang.Class.getDeclaredMethods0(Native Method)
0:46.88 at java.lang.Class.privateGetDeclaredMethods(Class.java:2615)
0:46.88 at java.lang.Class.getDeclaredMethods(Class.java:1860)
0:46.88 at org.mozilla.gecko.annotationProcessors.utils.GeneratableElementIterator.<init>(GeneratableElementIterator.java:32)
0:46.88 at org.mozilla.gecko.annotationProcessors.AnnotationProcessor.main(AnnotationProcessor.java:74)
0:46.88 Caused by: java.lang.ClassNotFoundException: android.support.v7.widget.RecyclerView$LayoutManager
0:46.88 at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
0:46.88 at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
0:46.88 at java.security.AccessController.doPrivileged(Native Method)
0:46.88 at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
0:46.88 at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
0:46.88 at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
0:46.88 ... 5 more
Is there something to do more in moz.build?
There is Makefile.in file in /mozilla-central/mobile/android/base/
Find ALL_JARS and add your external jar file there!
I have written a code to expand shortened url. The code works fine as a standalone program. But when I put it in Hadoop where I extract the url in Map function, I get the following error. I have mentioned all httpclient dependencies in the classpath while compiling the code. Please help me.
14/07/28 08:16:40 INFO mapreduce.Job: Task Id : attempt_1405534657345_0008_m_000001_2, Status : FAILED
Error: java.lang.ClassNotFoundException: org.apache.http.impl.client.HttpClientBuilder
at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
at org.myorg.ExtractUrl.unshortenSingleLevel(ExtractUrl.java:80)
at org.myorg.ExtractUrl.unshorten(ExtractUrl.java:118)
at org.myorg.ExtractUrl$Map.map(ExtractUrl.java:40)
at org.myorg.ExtractUrl$Map.map(ExtractUrl.java:27)
at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:430)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:342)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1548)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:163)
I am copying my external jars to /home/hadoop/lib directoy in emr as a bootstrap process. But it is showing following error during bootstrap process
Exception in thread "main" java.lang.IncompatibleClassChangeError: class com.google.common.cache.CacheBuilder$3 has interface com.google.common.base.Ticker as super class
at java.lang.ClassLoader.defineClass1(Native Method)
at java.lang.ClassLoader.defineClass(ClassLoader.java:792)
at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
at java.net.URLClassLoader.defineClass(URLClassLoader.java:449)
at java.net.URLClassLoader.access$100(URLClassLoader.java:71)
at java.net.URLClassLoader$1.run(URLClassLoader.java:361)
at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
at com.google.common.cache.CacheBuilder.<clinit>(CacheBuilder.java:207)
at org.apache.hadoop.security.ShellBasedUnixGroupsMapping.<clinit>(ShellBasedUnixGroupsMapping.java:46)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:270)
at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:861)
at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:906)
at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:932)
at org.apache.hadoop.security.Groups.<init>(Groups.java:48)
at org.apache.hadoop.security.Groups.getUserToGroupsMappingService(Groups.java:140)
at org.apache.hadoop.security.UserGroupInformation.initialize(UserGroupInformation.java:205)
at org.apache.hadoop.security.UserGroupInformation.ensureInitialized(UserGroupInformation.java:184)
at org.apache.hadoop.security.UserGroupInformation.isSecurityEnabled(UserGroupInformation.java:236)
at org.apache.hadoop.security.UserGroupInformation.getLoginUser(UserGroupInformation.java:477)
at org.apache.hadoop.security.UserGroupInformation.getCurrentUser(UserGroupInformation.java:463)
at org.apache.hadoop.fs.FileSystem$Cache$Key.<init>(FileSystem.java:1521)
at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:1422)
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:256)
at com.amazon.elasticmapreduce.scriptrunner.ScriptRunner.fetchFile(ScriptRunner.java:39)
at com.amazon.elasticmapreduce.scriptrunner.ScriptRunner.main(ScriptRunner.java:56)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.util.RunJar.main(RunJar.java:187)
Can anyone help me why it is happening?
I found the answer. The problem was that Mahout 0.7 is using an older version of Guava. I replaced Mahout 0.7 with 0.8 and it worked.