Im getting the following exception when attempting to connect to a derby server via openjpa:
attempted duplicate class definition for name: "sun/security/pkcs11/SunPKCS11". Looking through some google results it looks like theres a known problem with java on mac, but im using ubuntu, and openjdk-6. Im really not sure where to start solving this. Anyone any ideas whats going wrong?
java -version output:
java version "1.6.0_24"
OpenJDK Runtime Environment (IcedTea6 1.11.1) (6b24-1.11.1-4ubuntu3)
OpenJDK 64-Bit Server VM (build 20.0-b12, mixed mode)
Error stack trace (lengthy...)
Exception in thread "main" <openjpa-2.2.0-r422266:1244990 nonfatal general error> org.apache.openjpa.persistence.PersistenceException: There were errors initializing your configuration: java.lang.LinkageError: loader (instance of sun/misc/Launcher$ExtClassLoader): attempted duplicate class definition for name: "sun/security/pkcs11/SunPKCS11"
at java.lang.ClassLoader.defineClass1(Native Method)
at java.lang.ClassLoader.defineClass(ClassLoader.java:634)
at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
at java.net.URLClassLoader.defineClass(URLClassLoader.java:277)
at java.net.URLClassLoader.access$000(URLClassLoader.java:73)
at java.net.URLClassLoader$1.run(URLClassLoader.java:212)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:205)
at java.lang.ClassLoader.loadClass(ClassLoader.java:321)
at java.lang.ClassLoader.loadClass(ClassLoader.java:314)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:294)
at java.lang.ClassLoader.loadClass(ClassLoader.java:266)
at sun.security.jca.ProviderConfig$3.run(ProviderConfig.java:253)
at sun.security.jca.ProviderConfig$3.run(ProviderConfig.java:244)
at java.security.AccessController.doPrivileged(Native Method)
at sun.security.jca.ProviderConfig.doLoadProvider(ProviderConfig.java:244)
at sun.security.jca.ProviderConfig.getProvider(ProviderConfig.java:224)
at sun.security.jca.ProviderList.loadAll(ProviderList.java:281)
at sun.security.jca.ProviderList.removeInvalid(ProviderList.java:298)
at sun.security.jca.Providers.getFullProviderList(Providers.java:170)
at java.security.Security.getProviders(Security.java:457)
at java.security.Security.getProviders(Security.java:607)
at java.security.Security.getProviders(Security.java:552)
at org.apache.derby.client.am.EncryptionManager.<init>(Unknown Source)
at org.apache.derby.jdbc.ClientBaseDataSource.<clinit>(Unknown Source)
at org.apache.derby.jdbc.ClientDriver.tokenizeURLProperties(Unknown Source)
at org.apache.derby.jdbc.ClientDriver.connect(Unknown Source)
at org.apache.commons.dbcp.DriverConnectionFactory.createConnection(DriverConnectionFactory.java:38)
at org.apache.commons.dbcp.PoolableConnectionFactory.makeObject(PoolableConnectionFactory.java:582)
at org.apache.commons.dbcp.BasicDataSource.validateConnectionFactory(BasicDataSource.java:1556)
at org.apache.commons.dbcp.BasicDataSource.createPoolableConnectionFactory(BasicDataSource.java:1545)
at org.apache.commons.dbcp.BasicDataSource.createDataSource(BasicDataSource.java:1388)
at org.apache.commons.dbcp.BasicDataSource.getConnection(BasicDataSource.java:1044)
at org.apache.openjpa.jdbc.schema.DBCPDriverDataSource.getDBCPConnection(DBCPDriverDataSource.java:74)
at org.apache.openjpa.jdbc.schema.AutoDriverDataSource.getConnection(AutoDriverDataSource.java:42)
at org.apache.openjpa.jdbc.schema.SimpleDriverDataSource.getConnection(SimpleDriverDataSource.java:75)
at org.apache.openjpa.lib.jdbc.DelegatingDataSource.getConnection(DelegatingDataSource.java:118)
at org.apache.openjpa.lib.jdbc.DecoratingDataSource.getConnection(DecoratingDataSource.java:93)
at org.apache.openjpa.jdbc.schema.DataSourceFactory.installDBDictionary(DataSourceFactory.java:233)
at org.apache.openjpa.jdbc.conf.JDBCConfigurationImpl.getConnectionFactory(JDBCConfigurationImpl.java:733)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:616)
at org.apache.openjpa.lib.conf.ConfigurationImpl.instantiateAll(ConfigurationImpl.java:295)
at org.apache.openjpa.conf.OpenJPAConfigurationImpl.instantiateAll(OpenJPAConfigurationImpl.java:1671)
at org.apache.openjpa.kernel.AbstractBrokerFactory.makeReadOnly(AbstractBrokerFactory.java:646)
at org.apache.openjpa.kernel.AbstractBrokerFactory.newBroker(AbstractBrokerFactory.java:203)
at org.apache.openjpa.kernel.DelegatingBrokerFactory.newBroker(DelegatingBrokerFactory.java:156)
at org.apache.openjpa.persistence.EntityManagerFactoryImpl.createEntityManager(EntityManagerFactoryImpl.java:227)
at org.apache.openjpa.persistence.EntityManagerFactoryImpl.createEntityManager(EntityManagerFactoryImpl.java:154)
at org.apache.openjpa.persistence.EntityManagerFactoryImpl.createEntityManager(EntityManagerFactoryImpl.java:60)
at org.dynamosoftware.jpatest.App.main(App.java:15)
Related
I`m using spring boot 2.6.6 and logback and slf4j. when I run application in my local computer in IntellijA IDE using "Run configuration", application starts normally. but when I build app with goal "clean package" and then run with command line "java -jar myApp.war" the following error occures:
SLF4J: The requested version 1.7.16 by your slf4j binding is not compatible with [1.6]
SLF4J: See http://www.slf4j.org/codes.html#version_mismatch for further details.
Exception in thread "main" java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at org.springframework.boot.loader.MainMethodRunner.run(MainMethodRunner.java:49)
at org.springframework.boot.loader.Launcher.launch(Launcher.java:108)
at org.springframework.boot.loader.Launcher.launch(Launcher.java:58)
at org.springframework.boot.loader.WarLauncher.main(WarLauncher.java:59)
Caused by: java.lang.IllegalArgumentException: Cannot instantiate interface org.springframework.context.ApplicationListener : org.springframework.boot.context.logging.LoggingApplicationListener
at org.springframework.boot.SpringApplication.createSpringFactoriesInstances(SpringApplication.java:456)
at org.springframework.boot.SpringApplication.getSpringFactoriesInstances(SpringApplication.java:438)
at org.springframework.boot.SpringApplication.getSpringFactoriesInstances(SpringApplication.java:431)
at org.springframework.boot.SpringApplication.<init>(SpringApplication.java:263)
at org.springframework.boot.SpringApplication.<init>(SpringApplication.java:241)
at org.springframework.boot.SpringApplication.run(SpringApplication.java:1312)
at org.springframework.boot.SpringApplication.run(SpringApplication.java:1301)
at com.fanap.kidzy.vidzy.VidzyApplication.main(VidzyApplication.java:16)
... 8 more
Caused by: java.lang.NoSuchMethodError: org.springframework.util.Assert.isTrue(ZLjava/util/function/Supplier;)V
at org.springframework.core.ResolvableType.forClassWithGenerics(ResolvableType.java:1088)
at org.springframework.core.ResolvableType.forClassWithGenerics(ResolvableType.java:1074)
at org.springframework.boot.context.properties.bind.Bindable.mapOf(Bindable.java:267)
at org.springframework.boot.context.logging.LoggingApplicationListener.<clinit>(LoggingApplicationListener.java:98)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(Unknown Source)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(Unknown Source)
at java.lang.reflect.Constructor.newInstance(Unknown Source)
at org.springframework.beans.BeanUtils.instantiateClass(BeanUtils.java:126)
at org.springframework.boot.SpringApplication.createSpringFactoriesInstances(SpringApplication.java:452)
... 15 more
I analyzed the pom with "dependency analyzer" plugin but slf4j 1.7.6 or 1.6 not exists!
I have to get the coverage for my integration tests for spring boot app. Now when I am trying to use javaagent in startup for my spring boot app I am getting an error . Also I have used the below args in JAVA_OPTS
-javaagent:/tmp/jacocoagent.jar=destfile=/mnt/shared/jacoco.exec
at org.jacoco.agent.rt.internal_035b120.CoverageTransformer.transform(CoverageTransformer.java:93)
at sun.instrument.TransformerManager.transform(TransformerManager.java:188)
at sun.instrument.InstrumentationImpl.transform(InstrumentationImpl.java:428)
at java.lang.ClassLoader.defineClass1(Native Method)
at java.lang.ClassLoader.defineClass(ClassLoader.java:763)
at sun.reflect.GeneratedMethodAccessor20.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.springframework.cglib.core.ReflectUtils.defineClass(ReflectUtils.java:459)
at org.springframework.cglib.core.AbstractClassGenerator.generate(AbstractClassGenerator.java:336)
at org.springframework.cglib.core.AbstractClassGenerator$ClassLoaderData$3.apply(AbstractClassGenerator.java:93)
Caused by: java.io.IOException: Error while instrumenting xxx/xxx/core/config/CoreAutoConfiguration$$FastClassBySpringCGLIB$$e1df4252.
at org.jacoco.agent.rt.internal_035b120.core.instr.Instrumenter.instrumentError(Instrumenter.java:158)
at org.jacoco.agent.rt.internal_035b120.core.instr.Instrumenter.instrument(Instrumenter.java:108)
at org.jacoco.agent.rt.internal_035b120.CoverageTransformer.transform(CoverageTransformer.java:91)
... 52 more
Caused by: java.lang.IllegalStateException: Cannot process instrumented class xxx/xxx/core/config/CoreAutoConfiguration$$FastClassBySpringCGLIB$$e1df4252. Please supply original non-instrumented classes.
at org.jacoco.agent.rt.internal_035b120.core.internal.instr.InstrSupport.assertNotInstrumented(InstrSupport.java:237)
at org.jacoco.agent.rt.internal_035b120.core.internal.instr.ClassInstrumenter.visitField(ClassInstrumenter.java:55)
at org.jacoco.agent.rt.internal_035b120.asm.ClassVisitor.visitField(ClassVisitor.java:287)
at org.jacoco.agent.rt.internal_035b120.asm.ClassReader.readField(ClassReader.java:906)
at org.jacoco.agent.rt.internal_035b120.asm.ClassReader.accept(ClassReader.java:683)
at org.jacoco.agent.rt.internal_035b120.asm.ClassReader.accept(ClassReader.java:400)
at org.jacoco.agent.rt.internal_035b120.core.instr.Instrumenter.instrument(Instrumenter.java:88)
at org.jacoco.agent.rt.internal_035b120.core.instr.Instrumenter.instrument(Instrumenter.java:106)
... 53 more
java.lang.instrument.IllegalClassFormatException: Error while instrumenting xxx/xxx/core/config/CoreAutoConfiguration$$EnhancerBySpringCGLIB$$e954c394$$FastClassBySpringCGLIB$$3d9ac05.
at org.jacoco.agent.rt.internal_035b120.CoverageTransformer.transform(CoverageTransformer.java:93)
at sun.instrument.TransformerManager.transform(TransformerManager.java:188)
at sun.instrument.InstrumentationImpl.transform(InstrumentationImpl.java:428)
at java.lang.ClassLoader.defineClass1(Native Method)
at java.lang.ClassLoader.defineClass(ClassLoader.java:763)
at sun.reflect.GeneratedMethodAccessor20.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.springframework.cglib.core.ReflectUtils.defineClass(ReflectUtils.java:459)
at org.springframework.cglib.core.AbstractClassGenerator.generate(AbstractClassGenerator.java:336)
at org.springframework.cglib.core.AbstractClassGenerator$ClassLoaderData$3.apply(AbstractClassGenerator.java:93)
Caused by: java.io.IOException: Error while instrumenting xxx/xxx/core/config/CoreAutoConfiguration$$EnhancerBySpringCGLIB$$e954c394$$FastClassBySpringCGLIB$$3d9ac05.
at org.jacoco.agent.rt.internal_035b120.core.instr.Instrumenter.instrumentError(Instrumenter.java:158)
at org.jacoco.agent.rt.internal_035b120.core.instr.Instrumenter.instrument(Instrumenter.java:108)
at org.jacoco.agent.rt.internal_035b120.CoverageTransformer.transform(CoverageTransformer.java:91)
... 52 more
Caused by: java.lang.IllegalStateException: Cannot process instrumented class xxx/xxx/core/config/CoreAutoConfiguration$$EnhancerBySpringCGLIB$$e954c394$$FastClassBySpringCGLIB$$3d9ac05. Please supply original non-instrumented classes.
at org.jacoco.agent.rt.internal_035b120.core.internal.instr.InstrSupport.assertNotInstrumented(InstrSupport.java:237)
at org.jacoco.agent.rt.internal_035b120.core.internal.instr.ClassInstrumenter.visitField(ClassInstrumenter.java:55)
at org.jacoco.agent.rt.internal_035b120.asm.ClassVisitor.visitField(ClassVisitor.java:287)
at org.jacoco.agent.rt.internal_035b120.asm.ClassReader.readField(ClassReader.java:906)
at org.jacoco.agent.rt.internal_035b120.asm.ClassReader.accept(ClassReader.java:683)
at org.jacoco.agent.rt.internal_035b120.asm.ClassReader.accept(ClassReader.java:400)
at org.jacoco.agent.rt.internal_035b120.core.instr.Instrumenter.instrument(Instrumenter.java:88)
I'm trying to create a conecction from pentaho (7.0.0.0-25) to Jena (fuseki 2.4.1), using "connection type: SPARKSQL" (and localhost, port, and DBname) in pentaho, and I get this error:
Error connecting to database [test] :org.pentaho.di.core.exception.KettleDatabaseException:
Error occurred while trying to connect to the database
Error connecting to database: (using class org.apache.hive.jdbc.SparkSqlSimbaDriver)
No suitable driver found for jdbc:spark://localhost:3030/testSKMO;AuthMech=0;SocketTimeout=10
org.pentaho.di.core.exception.KettleDatabaseException:
Error occurred while trying to connect to the database
Error connecting to database: (using class org.apache.hive.jdbc.SparkSqlSimbaDriver)
No suitable driver found for jdbc:spark://localhost:3030/testSKMO;AuthMech=0;SocketTimeout=10
at org.pentaho.di.core.database.Database.normalConnect(Database.java:472)
at org.pentaho.di.core.database.Database.connect(Database.java:370)
at org.pentaho.di.core.database.Database.connect(Database.java:341)
at org.pentaho.di.core.database.Database.connect(Database.java:331)
at org.pentaho.di.core.database.DatabaseFactory.getConnectionTestReport(DatabaseFactory.java:80)
at org.pentaho.di.core.database.DatabaseMeta.testConnection(DatabaseMeta.java:2795)
at org.pentaho.ui.database.event.DataHandler.testDatabaseConnection(DataHandler.java:598)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.pentaho.ui.xul.impl.AbstractXulDomContainer.invoke(AbstractXulDomContainer.java:313)
at org.pentaho.ui.xul.impl.AbstractXulComponent.invoke(AbstractXulComponent.java:157)
at org.pentaho.ui.xul.impl.AbstractXulComponent.invoke(AbstractXulComponent.java:141)
at org.pentaho.ui.xul.swt.tags.SwtButton.access$500(SwtButton.java:43)
at org.pentaho.ui.xul.swt.tags.SwtButton$4.widgetSelected(SwtButton.java:137)
at org.eclipse.swt.widgets.TypedListener.handleEvent(Unknown Source)
at org.eclipse.swt.widgets.EventTable.sendEvent(Unknown Source)
at org.eclipse.swt.widgets.Display.sendEvent(Unknown Source)
at org.eclipse.swt.widgets.Widget.sendEvent(Unknown Source)
at org.eclipse.swt.widgets.Display.runDeferredEvents(Unknown Source)
at org.eclipse.swt.widgets.Display.readAndDispatch(Unknown Source)
at org.eclipse.jface.window.Window.runEventLoop(Window.java:820)
at org.eclipse.jface.window.Window.open(Window.java:796)
at org.pentaho.di.ui.xul.KettleDialog.show(KettleDialog.java:80)
at org.pentaho.di.ui.xul.KettleDialog.show(KettleDialog.java:47)
at org.pentaho.di.ui.core.database.dialog.XulDatabaseDialog.open(XulDatabaseDialog.java:116)
at org.pentaho.di.ui.core.database.dialog.DatabaseDialog.open(DatabaseDialog.java:60)
at org.pentaho.di.ui.spoon.delegates.SpoonDBDelegate.newConnection(SpoonDBDelegate.java:475)
at org.pentaho.di.ui.spoon.delegates.SpoonDBDelegate.newConnection(SpoonDBDelegate.java:462)
at org.pentaho.di.ui.spoon.Spoon.doubleClickedInTree(Spoon.java:3066)
at org.pentaho.di.ui.spoon.Spoon.doubleClickedInTree(Spoon.java:3036)
at org.pentaho.di.ui.spoon.Spoon.access$2200(Spoon.java:361)
at org.pentaho.di.ui.spoon.Spoon$26.widgetDefaultSelected(Spoon.java:6169)
at org.eclipse.swt.widgets.TypedListener.handleEvent(Unknown Source)
at org.eclipse.swt.widgets.EventTable.sendEvent(Unknown Source)
at org.eclipse.swt.widgets.Display.sendEvent(Unknown Source)
at org.eclipse.swt.widgets.Widget.sendEvent(Unknown Source)
at org.eclipse.swt.widgets.Display.runDeferredEvents(Unknown Source)
at org.eclipse.swt.widgets.Display.readAndDispatch(Unknown Source)
at org.pentaho.di.ui.spoon.Spoon.readAndDispatch(Spoon.java:1359)
at org.pentaho.di.ui.spoon.Spoon.waitForDispose(Spoon.java:7990)
at org.pentaho.di.ui.spoon.Spoon.start(Spoon.java:9290)
at org.pentaho.di.ui.spoon.Spoon.main(Spoon.java:685)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.pentaho.commons.launcher.Launcher.main(Launcher.java:92)
Caused by: org.pentaho.di.core.exception.KettleDatabaseException:
Error connecting to database: (using class org.apache.hive.jdbc.SparkSqlSimbaDriver)
No suitable driver found for jdbc:spark://localhost:3030/testSKMO;AuthMech=0;SocketTimeout=10
at org.pentaho.di.core.database.Database.connectUsingClass(Database.java:585)
at org.pentaho.di.core.database.Database.normalConnect(Database.java:456)
... 48 more
Caused by: java.sql.SQLException: No suitable driver found for jdbc:spark://localhost:3030/testSKMO;AuthMech=0;SocketTimeout=10
at java.sql.DriverManager.getConnection(DriverManager.java:689)
at java.sql.DriverManager.getConnection(DriverManager.java:270)
at org.pentaho.di.core.database.Database.connectUsingClass(Database.java:571)
... 49 more
Hostname :localhost
Port :3030
Database name :testSKMO
I have download .jar files "jena-jdbc-driver-bundle-3.1.0.jar" (contains driver to Jena connection) and "pentaho-big-data-kettle-plugins-hive.jar" (contains "SparkSQLSimbaDriver" class, that class looks like cause error) and I have tried somethings:
To put that .jar in Pentaho/data-integration/lib
To create a new folder Pentaho/data-integration/libext/JDBC and to put .jar in this.
In both cases still getting the same error.
Someone have any idea or hint to solve it??
Finally, the question is how I can connect Pentaho with Jena.
Thanks!
To connect Pentaho with Jena you must:
put "jena-jdbc-driver-bundle-3.1.0.jar" (it contains drivers to connect with Jena) in folder Pentaho/data-integration/lib/
use a connection type "Generic" (in Pentaho, when you create a new connection). Probably, that is the problem's key
define, in this connection, "jdbc:jena:remote:update=http://localhost:3030/name_of_dataset/update" (in my case) like custom connection URL and "org.apache.jena.jdbc.remote.RemoteEndpointDriver" like driver class
...and... go!!
I am trying to run Oracle's Object Type Translator Utility (OTT) and getting below error:
Exception in thread "main" java.lang.UnsatisfiedLinkError: C:\oraclexe\app\oracle\product\11.2.0\server\bin\ocijdbc11.dll: Can't load AMD 64-bit .dll on a IA 32-bit platform
at java.lang.ClassLoader$NativeLibrary.load(Native Method)
at java.lang.ClassLoader.loadLibrary0(Unknown Source)
at java.lang.ClassLoader.loadLibrary(Unknown Source)
at java.lang.Runtime.loadLibrary0(Unknown Source)
at java.lang.System.loadLibrary(Unknown Source)
at oracle.jdbc.driver.T2CConnection$1.run(T2CConnection.java:3516)
at java.security.AccessController.doPrivileged(Native Method)
at oracle.jdbc.driver.T2CConnection.loadNativeLibrary(T2CConnection.java:3512)
at oracle.jdbc.driver.T2CConnection.logon(T2CConnection.java:266)
at oracle.jdbc.driver.PhysicalConnection.<init>(PhysicalConnection.java:536)
at oracle.jdbc.driver.T2CConnection.<init>(T2CConnection.java:162)
at oracle.jdbc.driver.T2CDriverExtension.getConnection(T2CDriverExtension.java:53)
at oracle.jdbc.driver.OracleDriver.connect(OracleDriver.java:521)
at java.sql.DriverManager.getConnection(Unknown Source)
at java.sql.DriverManager.getConnection(Unknown Source)
at oracle.ott.Konnection.getTheConnection(Konnection.java:102)
at oracle.ott.Konnection.<init>(Konnection.java:39)
at oracle.ott.Doit.main(Doit.java:107)
at oracle.ott.c.CMain.main(CMain.java:9)
It was working fine few months ago and no idea why this is giving error now.
So, I figured out finally that there was some issue with the old setup of instant client 11_2.0, so i installed new instant client 11_2.2 and set all the required environment variables and it works perfectly fine.
We are using bdutil 1.1 to deploy a Spark (1.2.0) cluster. However, we are having an issue when we launch our spark script:
py4j.protocol.Py4JJavaError: An error occurred while calling o70.registerTempTable.
: java.lang.RuntimeException: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.metastore.HiveMetaStoreClient
at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:346)
at org.apache.spark.sql.hive.HiveContext$$anonfun$4.apply(HiveContext.scala:235)
at org.apache.spark.sql.hive.HiveContext$$anonfun$4.apply(HiveContext.scala:231)
at scala.Option.orElse(Option.scala:257)
at org.apache.spark.sql.hive.HiveContext.x$3$lzycompute(HiveContext.scala:231)
at org.apache.spark.sql.hive.HiveContext.x$3(HiveContext.scala:229)
at org.apache.spark.sql.hive.HiveContext.hiveconf$lzycompute(HiveContext.scala:229)
at org.apache.spark.sql.hive.HiveContext.hiveconf(HiveContext.scala:229)
at org.apache.spark.sql.hive.HiveMetastoreCatalog.<init>(HiveMetastoreCatalog.scala:54)
at org.apache.spark.sql.hive.HiveContext$$anon$1.<init>(HiveContext.scala:253)
at org.apache.spark.sql.hive.HiveContext.catalog$lzycompute(HiveContext.scala:253)
at org.apache.spark.sql.hive.HiveContext.catalog(HiveContext.scala:253)
at org.apache.spark.sql.hive.HiveContext.catalog(HiveContext.scala:72)
at org.apache.spark.sql.SQLContext.registerRDDAsTable(SQLContext.scala:279)
at org.apache.spark.sql.SchemaRDDLike$class.registerTempTable(SchemaRDDLike.scala:86)
at org.apache.spark.sql.api.java.JavaSchemaRDD.registerTempTable(JavaSchemaRDD.scala:42)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:231)
at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:379)
at py4j.Gateway.invoke(Gateway.java:259)
at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:133)
at py4j.commands.CallCommand.execute(CallCommand.java:79)
at py4j.GatewayConnection.run(GatewayConnection.java:207)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.metastore.HiveMetaStoreClient
at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1412)
at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:62)
at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:72)
at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:2453)
at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:2465)
at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:340)
... 26 more
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1410)
... 31 more
Caused by: javax.jdo.JDOFatalUserException: Class org.datanucleus.api.jdo.JDOPersistenceManagerFactory was not found.
The script works on my laptop. I have datanucleus-api-jdo-3.2.6.jar in the /home/hadoop/spark-install/lib path.
Any ideas of what could be wrong?
I needed to add :
SPARK_CLASSPATH=/home/hadoop/spark-install/lib/datanucleus-api-jdo-3.2.6.jar:/home/hadoop/spark-install/lib/datanucleus-core-3.2.10.jar:/home/hadoop/spark-install/lib/datanucleus-rdbms-3.2.9.jar
before my spark-submit command:
SPARK_CLASSPATH=/home/hadoop/spark-install/lib/datanucleus-api-jdo-3.2.6.jar:/home/hadoop/spark-install/lib/datanucleus-core-3.2.10.jar:/home/hadoop/spark-install/lib/datanucleus-rdbms-3.2.9.jar ../hadoop/spark-install/bin/spark-submit main.py --master spark://spark-m:7077