Maven dependency conflicts during test (surefire) but not production - maven

I have an application which requires a couple different versions of the same libraries. When running through intellij this isn't a problem because it exports the classpath in a way that avoid the conflicts. I then use the maven shade plugin to create an uber-jar when compiling for production.
However when using the surefire plugin to run my unit tests the conflicts appear again. Does anyone know a way that I can ensure that the sure-fire plugin creates the class path in a way to avoid these conflicts?
Thanks!
UPDATE: Below is the error I am getting (Although it is fairly specific to my application). Basically there are two versions of Saxon HE and PE which are both being loaded. When running the surefire tests it causes this exception:
java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at com.jorsek.appserver.bootstrap.Main.invokeMain(Main.java:130)
at com.jorsek.appserver.bootstrap.Main.run(Main.java:449)
at framework.TestUtils.startupDatabase(TestUtils.java:41)
at framework.AbstractTestInstance.startDatabase(AbstractTestInstance.java:41)
at framework.AbstractTestInstance.startDatabase(AbstractTestInstance.java:55)
at org.jorsek.exist.versioning.AllSetslTest.setUpBeforeClass(AllSetslTest.java:80)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:24)
at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
at org.apache.maven.surefire.junit4.JUnit4TestSet.execute(JUnit4TestSet.java:59)
at org.apache.maven.surefire.suite.AbstractDirectoryTestSuite.executeTestSet(AbstractDirectoryTestSuite.java:120)
at org.apache.maven.surefire.suite.AbstractDirectoryTestSuite.execute(AbstractDirectoryTestSuite.java:103)
at org.apache.maven.surefire.Surefire.run(Surefire.java:169)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.maven.surefire.booter.SurefireBooter.runSuitesInProcess(SurefireBooter.java:350)
at org.apache.maven.surefire.booter.SurefireBooter.main(SurefireBooter.java:1021)
Caused by: java.util.ServiceConfigurationError: javax.xml.transform.TransformerFactory: Provider com.saxonica.config.ProfessionalTransformerFactory could not be instantiated: java.lang.VerifyError: (class: com/saxonica/config/ProfessionalConfiguration, method: newExpressionParser signature: (Ljava/lang/String;ZLnet/sf/saxon/value/DecimalValue;)Lnet/sf/saxon/expr/ExpressionParser;) Wrong return type in function
at java.util.ServiceLoader.fail(ServiceLoader.java:207)
at java.util.ServiceLoader.access$100(ServiceLoader.java:164)
at java.util.ServiceLoader$LazyIterator.next(ServiceLoader.java:360)
at java.util.ServiceLoader$1.next(ServiceLoader.java:428)
at org.exist.validation.XmlLibraryChecker.check(XmlLibraryChecker.java:162)
at com.jorsek.appserver.jetty.JettyStart.<init>(JettyStart.java:95)
at com.jorsek.appserver.jetty.JettyStart.main(JettyStart.java:77)
... 30 more
Caused by: java.lang.VerifyError: (class: com/saxonica/config/ProfessionalConfiguration, method: newExpressionParser signature: (Ljava/lang/String;ZLnet/sf/saxon/value/DecimalValue;)Lnet/sf/saxon/expr/ExpressionParser;) Wrong return type in function
at com.saxonica.config.ProfessionalTransformerFactory.<init>(ProfessionalTransformerFactory.java:24)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
at java.lang.Class.newInstance0(Class.java:357)
at java.lang.Class.newInstance(Class.java:310)
at java.util.ServiceLoader$LazyIterator.next(ServiceLoader.java:356)
... 34 more

Perhaps you can override the other dependency on test scope as well? Or if need be, just override them both on default scope. Or is there some reason why you cannot do either of the above?
Or an alternative of course is to start dabbling with the classloader during your test runs, like shown here.

Related

Deploy with JRebel from IntellijIDEA

I am following a tutorial from the "Spring in Action" book (by Manning Pubblications).
I have a problem when I try to deploy a webApp (with springMVC) with JRebel.
[2016-04-04 11:00:28,040] Artifact WebAppSpring: Artifact is being deployed, please wait...
Connected to server
2016-04-04 11:00:28 JRebel: ERROR Class 'org.apache.catalina.startup.ContextConfig' could not be processed by org.zeroturnaround.javarebel.integration.catalina.ContextConfigCBP#java.net.URLClassLoader#1134affc: org.zeroturnaround.bundled.javassist.CannotCompileException: [source error] processAnnotationsUrl(java.net.URL,org.apache.tomcat.util.descriptor.web.WebXml,boolean) not found in org.apache.catalina.startup.ContextConfig
at org.zeroturnaround.bundled.javassist.CtNewMethod.make(JRebel:79)
at org.zeroturnaround.bundled.javassist.CtNewMethod.make(JRebel:45)
at org.zeroturnaround.bundled.javassist.CtMethod.make(JRebel:132)
at org.zeroturnaround.javarebel.integration.catalina.ContextConfigCBP.implementReloadingOnTomcat8(ContextConfigCBP.java:122)
at org.zeroturnaround.javarebel.integration.catalina.ContextConfigCBP.process(ContextConfigCBP.java:25)
at org.zeroturnaround.javarebel.integration.support.JavassistClassBytecodeProcessor.process(JRebel:117)
at com.zeroturnaround.javarebel.kh.a(JRebel:359)
at com.zeroturnaround.javarebel.kh.a(JRebel:348)
at com.zeroturnaround.javarebel.kh.a(JRebel:316)
at com.zeroturnaround.javarebel.SDKIntegrationImpl.runBytecodeProcessors(JRebel:132)
at com.zeroturnaround.javarebel.jP.transform(JRebel:57)
at java.lang.ClassLoader.defineClass(ClassLoader.java)
at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
at java.net.URLClassLoader.defineClass(URLClassLoader.java:467)
at java.net.URLClassLoader.access$100(URLClassLoader.java:73)
at java.net.URLClassLoader$1.run(URLClassLoader.java:368)
at java.net.URLClassLoader$1.run(URLClassLoader.java:362)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:361)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
at org.apache.catalina.mbeans.MBeanFactory.createStandardContext(MBeanFactory.java:462)
at org.apache.catalina.mbeans.MBeanFactory.createStandardContext(MBeanFactory.java:431)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at org.apache.tomcat.util.modeler.BaseModelMBean.invoke(BaseModelMBean.java:300)
at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.invoke(DefaultMBeanServerInterceptor.java:819)
at com.sun.jmx.mbeanserver.JmxMBeanServer.invoke(JmxMBeanServer.java:801)
at javax.management.remote.rmi.RMIConnectionImpl.doOperation(RMIConnectionImpl.java:1471)
at javax.management.remote.rmi.RMIConnectionImpl.access$300(RMIConnectionImpl.java:76)
at javax.management.remote.rmi.RMIConnectionImpl$PrivilegedOperation.run(RMIConnectionImpl.java:1312)
at javax.management.remote.rmi.RMIConnectionImpl.doPrivilegedOperation(RMIConnectionImpl.java:1404)
at javax.management.remote.rmi.RMIConnectionImpl.invoke(RMIConnectionImpl.java:832)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:323)
at sun.rmi.transport.Transport$1.run(Transport.java:200)
at sun.rmi.transport.Transport$1.run(Transport.java:197)
at java.security.AccessController.doPrivileged(Native Method)
at sun.rmi.transport.Transport.serviceCall(Transport.java:196)
at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:568)
at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:826)
at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.lambda$run$256(TCPTransport.java:683)
at java.security.AccessController.doPrivileged(Native Method)
at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:682)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: compile error: processAnnotationsUrl(java.net.URL,org.apache.tomcat.util.descriptor.web.WebXml,boolean) not found in org.apache.catalina.startup.ContextConfig
at org.zeroturnaround.bundled.javassist.compiler.TypeChecker.atMethodCallCore(JRebel:749)
at org.zeroturnaround.bundled.javassist.compiler.TypeChecker.atCallExpr(JRebel:695)
at org.zeroturnaround.bundled.javassist.compiler.JvstTypeChecker.atCallExpr(JRebel:157)
at org.zeroturnaround.bundled.javassist.compiler.ast.CallExpr.accept(JRebel:46)
at org.zeroturnaround.bundled.javassist.compiler.CodeGen.doTypeCheck(JRebel:242)
at org.zeroturnaround.bundled.javassist.compiler.CodeGen.atStmnt(JRebel:330)
at org.zeroturnaround.bundled.javassist.compiler.ast.Stmnt.accept(JRebel:50)
at org.zeroturnaround.bundled.javassist.compiler.CodeGen.atStmnt(JRebel:351)
at org.zeroturnaround.bundled.javassist.compiler.ast.Stmnt.accept(JRebel:50)
at org.zeroturnaround.bundled.javassist.compiler.CodeGen.atIfStmnt(JRebel:398)
at org.zeroturnaround.bundled.javassist.compiler.CodeGen.atStmnt(JRebel:355)
at org.zeroturnaround.bundled.javassist.compiler.ast.Stmnt.accept(JRebel:50)
at org.zeroturnaround.bundled.javassist.compiler.CodeGen.atStmnt(JRebel:351)
at org.zeroturnaround.bundled.javassist.compiler.ast.Stmnt.accept(JRebel:50)
at org.zeroturnaround.bundled.javassist.compiler.CodeGen.atIfStmnt(JRebel:398)
at org.zeroturnaround.bundled.javassist.compiler.CodeGen.atStmnt(JRebel:355)
at org.zeroturnaround.bundled.javassist.compiler.ast.Stmnt.accept(JRebel:50)
at org.zeroturnaround.bundled.javassist.compiler.CodeGen.atStmnt(JRebel:351)
at org.zeroturnaround.bundled.javassist.compiler.ast.Stmnt.accept(JRebel:50)
at org.zeroturnaround.bundled.javassist.compiler.CodeGen.atMethodBody(JRebel:292)
at org.zeroturnaround.bundled.javassist.compiler.CodeGen.atMethodDecl(JRebel:274)
at org.zeroturnaround.bundled.javassist.compiler.ast.MethodDecl.accept(JRebel:44)
at org.zeroturnaround.bundled.javassist.compiler.Javac.compileMethod(JRebel:169)
at org.zeroturnaround.bundled.javassist.compiler.Javac.compile(JRebel:95)
at org.zeroturnaround.bundled.javassist.CtNewMethod.make(JRebel:74)
... 51 more
2016-04-04 11:00:32 JRebel: Directory 'C:\Users\NINO\IdeaProjects\SpringInAction\out\production\SpringInAction' will be monitored for changes.
2016-04-04 11:00:32 JRebel: Directory 'C:\Users\NINO\IdeaProjects\SpringInAction\web' will be monitored for changes.
...
...
...
After this error, the deploy go fine.
But if I modify a static file (jsp for example) jrebel works fine...if I modify a java class, jrebel not works.
I don't know if is a problem relative to the error shown in console at deploy time.
In the past I userd Jrebel with eclipse with servlet (not spring) and it worked fine.
I use:
- IntellijIDEA 2016.1.1
- Java 1.8
- Spring 4.2.5
Can you help me???
Thanks
What version of tomcat are you using? 9.0.0 milestone? Grab JRebel 6.4.2 release from here and install the zip using the same instructions as for installing the nightly build at here, should fix the error.

Error while executing Hive command line

I am getting the following error when tried to execute hive command line interface. May I know the incompatible Jar files causing the problem and the solution to fix it?
Exception in thread "main" java.lang.NoSuchFieldError: ALLOW_UNQUOTED_CONTROL_CHARS
at org.apache.hadoop.hive.ql.udf.generic.GenericUDTFJSONTuple.<clinit>(GenericUDTFJSONTuple.java:59)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:113)
at org.apache.hadoop.hive.ql.exec.FunctionRegistry.registerGenericUDTF(FunctionRegistry.java:545)
at org.apache.hadoop.hive.ql.exec.FunctionRegistry.registerGenericUDTF(FunctionRegistry.java:539)
at org.apache.hadoop.hive.ql.exec.FunctionRegistry.<clinit>(FunctionRegistry.java:472)
at org.apache.hadoop.hive.ql.session.SessionState.<init>(SessionState.java:202)
at org.apache.hadoop.hive.cli.CliSessionState.<init>(CliSessionState.java:86)
at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:635)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:613)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.RunJar.main(RunJar.java:156)

Unable to mock class in mockito

I am trying to write a unit test for a sonar plugin. However, on mocking its Project class, the test fails as Mockito tries to use a wrong classname.
my test code is:
import org.sonar.api.resources.Project;
Project project = mock(Project.class);
On running the unit test, either through maven or directly in eclipse, the following stacktrace comes up:
java.lang.NoClassDefFoundError: org/apache/maven/project/MavenProject
at java.lang.Class.getDeclaredMethods0(Native Method)
at java.lang.Class.privateGetDeclaredMethods(Class.java:2427)
at java.lang.Class.getDeclaredMethods(Class.java:1791)
at org.mockito.cglib.core.ReflectUtils.addAllMethods(ReflectUtils.java:349)
at org.mockito.cglib.proxy.Enhancer.getMethods(Enhancer.java:422)
at org.mockito.cglib.proxy.Enhancer.generateClass(Enhancer.java:457)
at org.mockito.cglib.core.DefaultGeneratorStrategy.generate(DefaultGeneratorStrategy.java:25)
at org.mockito.cglib.core.AbstractClassGenerator.create(AbstractClassGenerator.java:217)
at org.mockito.cglib.proxy.Enhancer.createHelper(Enhancer.java:378)
at org.mockito.cglib.proxy.Enhancer.createClass(Enhancer.java:318)
at org.mockito.internal.creation.jmock.ClassImposterizer.createProxyClass(ClassImposterizer.java:93)
at org.mockito.internal.creation.jmock.ClassImposterizer.imposterise(ClassImposterizer.java:50)
at org.mockito.internal.util.MockUtil.createMock(MockUtil.java:54)
at org.mockito.internal.MockitoCore.mock(MockitoCore.java:45)
at org.mockito.Mockito.mock(Mockito.java:921)
at org.mockito.Mockito.mock(Mockito.java:816)
at com.databorough.sonar.xametric.MetricAnalysisSensorTest.shouldExecuteOnProjectWithAllParams(MetricAnalysisSensorTest.java:37)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:44)
at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:41)
at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28)
at org.junit.runners.BlockJUnit4ClassRunner.runNotIgnored(BlockJUnit4ClassRunner.java:79)
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:71)
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:49)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:193)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:52)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:191)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:42)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:184)
at org.junit.runners.ParentRunner.run(ParentRunner.java:236)
at org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:252)
at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:141)
at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:112)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray(ReflectionUtils.java:189)
at org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invoke(ProviderFactory.java:165)
at org.apache.maven.surefire.booter.ProviderFactory.invokeProvider(ProviderFactory.java:85)
at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:113)
at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:75)
Caused by: java.lang.ClassNotFoundException: org.apache.maven.project.MavenProject
at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
... 47 more
The project is in the classpath as the plugin builds successfully.
How can I set the test runner/maven to treat Project as org.sonar.api.resources.Project class?
Your "Project" class is correctly treated as "org.sonar.api.resources.Project".
The problem comes from the fact that Sonar internally uses "org.apache.maven.project.MavenProject", so this class must be on your test classpath. Adding the following dependency to your POM will solve the issue:
<dependency>
<groupId>org.apache.maven</groupId>
<artifactId>maven-project</artifactId>
<version>2.0.6</version>
<scope>test</scope>
</dependency>
java.lang.NoClassDefFoundError: org/apache/maven/project/MavenProject
this suggests that class org.sonar.api.resources.Project maybe depending on Maven class, org.apache.maven.project.MavenProject
I am not sure whether Sonar adds this as a dependency? You need to add dependency for Maven Plugin API maybe.

HBase completebulkload returns exception

I am trying to bulk-populate an HBase table quickly from a text file (several GB) by using the bulk loading method described in the Hadoop docs.
I have created an HFile which I now want to push to my HBase table.
When I use this command:
hadoop jar /home/hxcaine/hadoop/lib/hbase.jar completebulkload /user/hxcaine/dbpopulate/output/cf1 my_hbase_table
The job starts and then I get this exception:
Exception in thread "main" java.lang.NoClassDefFoundError: com/google/common/util/concurrent/ThreadFactoryBuilder
at org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles.doBulkLoad(LoadIncrementalHFiles.java:195)
at org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles.run(LoadIncrementalHFiles.java:696)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
at org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles.main(LoadIncrementalHFiles.java:701)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:68)
at org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:139)
at org.apache.hadoop.hbase.mapreduce.Driver.main(Driver.java:49)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.RunJar.main(RunJar.java:197)
Caused by: java.lang.ClassNotFoundException: com.google.common.util.concurrent.ThreadFactoryBuilder
at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
... 17 more
However, I can see that the Guava jar is in my classpath and when I check inside the jar I can see ThreadFactoryBuilder.class.
I am using these versions (and stuck with them):
Hadoop 0.20.2-cdh3u3
HBase 0.90.4-cdh3u3
Guava jar: /usr/lib/hadoop-0.20/lib/guava-r09-jarjar.jar
I do have an older Guava jar in my classpath but I don't know where it came from, I don't suppose it should have an effect.
Any ideas?
what happens if you run:
export HADOOP_CLASSPATH=`hbase classpath`
before running the load? From the stack trace, it looks like the jar is needed by one of the actual tasks though I am surprised to see that this actually kicks off an M/R job.

pig: java.lang.NoClassDefFoundError: org/jruby/embed/ScriptingContainer

pig 0.10.0 supports ruby UDF. So, i am trying a very simple example. but got the following error. Do you know why?
Pig Stack Trace
--------------- ERROR 2998: Unhandled internal error. org/jruby/embed/ScriptingContainer
java.lang.NoClassDefFoundError: org/jruby/embed/ScriptingContainer at
org.apache.pig.scripting.jruby.JrubyScriptEngine.<clinit>(JrubyScriptEngine.java:65)
at java.lang.Class.forName0(Native Method) at
java.lang.Class.forName(Class.java:169) at
org.apache.pig.scripting.ScriptEngine.getInstance(ScriptEngine.java:254)
at org.apache.pig.PigServer.registerCode(PigServer.java:523) at
org.apache.pig.tools.grunt.GruntParser.processRegister(GruntParser.java:422)
at
org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:419)
at
org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:189)
at
org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:165)
at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:84) at
org.apache.pig.Main.run(Main.java:555) at
org.apache.pig.Main.main(Main.java:111) at
sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597) at
org.apache.hadoop.util.RunJar.main(RunJar.java:156) Caused by:
java.lang.ClassNotFoundException: org.jruby.embed.ScriptingContainer
at java.net.URLClassLoader$1.run(URLClassLoader.java:202) at
java.security.AccessController.doPrivileged(Native Method) at
java.net.URLClassLoader.findClass(URLClassLoader.java:190) at
java.lang.ClassLoader.loadClass(ClassLoader.java:306) at
sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301) at
java.lang.ClassLoader.loadClass(ClassLoader.java:247) ... 17 more
================================================================================
I had the same problem. You should look to see if you have a jruby.jar installed w pig.
Seems like the jython.jar was there so maybe that's a friendly nudge for people to use python.
I had to explicitly put the jruby.jar in the classpath doing:
java -cp $PIG_HOME/pig-0.11.1.jar:$PIG_HOME/lib/jruby.jar org.apache.pig.Main -x local myscript.pig

Resources