Sqoop not working with Oozie (HUE) - sqoop

I'm trying to execute a sqoop command using Hue but fails. Part of the log is:
Sqoop command arguments :
--
import
--connect
jdbc:oracle:thin:#orps:1522:con12
--username
sqa
--password
********
--query
'SELECT
*
FROM
TARQ32
WHERE
$CONDITIONS'
--split-by
MO_NUMLINE
--as-sequencefile
--target-dir
/user/pes9/oracle/L12
-m
8
Fetching child yarn jobs
tag id : oozie-365c82c0bacd9831582dfe7edb354dfb
2017-11-13 15:47:50,208 [main] INFO org.apache.hadoop.yarn.client.RMProxy - Connecting to ResourceManager at xxx.yyyy.net/10.4.58.61:8032
Child yarn jobs are found -
=================================================================
>>> Invoking Sqoop command line now >>>
2017-11-13 15:47:50,387 [main] WARN org.apache.sqoop.tool.SqoopTool - $SQOOP_CONF_DIR has not been set in the environment. Cannot check for additional configuration.
<<< Invocation of Sqoop command completed <<<
No child hadoop job is executed.
Intercepting System.exit(1)
<<< Invocation of Main class completed <<<
Failing Oozie Launcher, Main class [org.apache.oozie.action.hadoop.SqoopMain], exit code [1]
Oozie Launcher failed, finishing Hadoop job gracefully
I tried to copy the ojdbc.jar in oozie lib, hdfs sharedlib, and other locations but nothing works. The same issue happend with Spark jobs executions, when I create a workflow and execute a spark job I'm getting thi error:
Failing Oozie Launcher, Main class [org.apache.oozie.action.hadoop.SparkMain], exit code [1]
Any idea?
[Update]
Stderr
Nov 13, 2017 3:47:44 PM com.google.inject.servlet.InternalServletModule$BackwardsCompatibleServletContextProvider get
WARNING: You are attempting to use a deprecated API (specifically, attempting to #Inject ServletContext inside an eagerly created singleton. While we allow this for backwards compatibility, be warned that this MAY have unexpected behavior if you have more than one injector (with ServletModule) running in the same JVM. Please consult the Guice documentation at http://code.google.com/p/google-guice/wiki/Servlets for more information.
Nov 13, 2017 3:47:44 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory register
INFO: Registering org.apache.hadoop.mapreduce.v2.app.webapp.JAXBContextResolver as a provider class
Nov 13, 2017 3:47:44 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory register
INFO: Registering org.apache.hadoop.yarn.webapp.GenericExceptionHandler as a provider class
Nov 13, 2017 3:47:44 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory register
INFO: Registering org.apache.hadoop.mapreduce.v2.app.webapp.AMWebServices as a root resource class
Nov 13, 2017 3:47:44 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.9 09/02/2011 11:17 AM'
Nov 13, 2017 3:47:44 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory getComponentProvider
INFO: Binding org.apache.hadoop.mapreduce.v2.app.webapp.JAXBContextResolver to GuiceManagedComponentProvider with the scope "Singleton"
Nov 13, 2017 3:47:45 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory getComponentProvider
INFO: Binding org.apache.hadoop.yarn.webapp.GenericExceptionHandler to GuiceManagedComponentProvider with the scope "Singleton"
Nov 13, 2017 3:47:45 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory getComponentProvider
INFO: Binding org.apache.hadoop.mapreduce.v2.app.webapp.AMWebServices to GuiceManagedComponentProvider with the scope "PerRequest"
log4j:WARN No appenders could be found for logger (org.apache.hadoop.mapreduce.v2.app.MRAppMaster).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.

Related

HDFS yarn nodemanager not starting completely

I am facing an issue from past few days,
If I use Hadoop 2.7.4, I am not able to start nodemanager's at slaves with this version, because of this I can't run any mapred jobs.
When I use Hadoop 2.8.2, everything is starting well(start-dfs.sh and start-yarn.sh) and I am able to see the content, nodes activity at http://:50070, but while running my programs that use data from HDFS, it keeps on displaying
17/11/15 12:51:46 WARN hdfs.DFSClient: zero
but the job runs well. I am not aware of this issue and how it is effected. So, I tried 2.7.3
When I tried 2.7.3, I am not facing the above error, but I see that the yarn are not starting completing and when I start yarn, it throws more line on screen that usual way and it starts. The main issue, in this case, I am not able to watch watch going on in hadoop from web url as here it doesn't display anything on web, except for those files present in HDFS(So, I am only able see the data from Utilities --> Browse files )
The error is similar to this:
starting yarn daemons
starting resourcemanager, logging to
/usr/local/hadoop/logs/yarn--resourcemanager-hadoop-master.out
Nov 20, 2017 8:01:28 AM
com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory
register
INFO: Registering
org.apache.hadoop.yarn.server.resourcemanager.webapp.JAXBContextResolver
as a provider class
Nov 20, 2017 8:01:28 AM
com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory
register
INFO: Registering
org.apache.hadoop.yarn.server.resourcemanager.webapp.RMWebServices as
a root resource class
Nov 20, 2017 8:01:28 AM
com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory
register
INFO: Registering
org.apache.hadoop.yarn.webapp.GenericExceptionHandler as a provider
class
Nov 20, 2017 8:01:28 AM
com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.9 09/02/2011
11:17 AM'
Nov 20, 2017 8:01:28 AM
com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory
getComponentProvider
INFO: Binding
org.apache.hadoop.yarn.server.resourcemanager.webapp.JAXBContextResolver
to GuiceManagedComponentProvider with the scope "Singleton"
hadoop-slave1: Warning: Permanently added 'hadoop-slave1,10.40.0.0'
(ECDSA) to the list of known hosts.
hadoop-slave1: starting nodemanager, logging to
/usr/local/hadoop/logs/yarn-root-nodemanager-hadoop-slave1.weave.local.out
Any idea how to set it up completely without any issue?

Getting error while running Webservices on Tomcat7 server - Ubuntu 16.04 LTS

Technical Environment:-
Ubuntu 16.04 LTS
Tomcat 7
Java 7
Eclipse Luna
I have imported a webservice into my eclipse luna workspace and tried to run that service on tomcat7 server. But I'm getting the following error.
Jul 24, 2016 5:39:30 PM org.apache.tomcat.util.digester.SetPropertiesRule begin
WARNING: [SetPropertiesRule]{Server/Service/Engine/Host/Context} Setting property 'source' to 'org.eclipse.jst.j2ee.server:service' did not find a matching property.
Jul 24, 2016 5:39:30 PM org.apache.catalina.startup.VersionLoggerListener log
INFO: Server version: Apache Tomcat/7.0.68 (Ubuntu)
Jul 24, 2016 5:39:30 PM org.apache.catalina.startup.VersionLoggerListener log
INFO: Server built: Jun 27 2016 18:13:17 UTC
Jul 24, 2016 5:39:30 PM org.apache.catalina.startup.VersionLoggerListener log
INFO: Server number: 7.0.68.0
Jul 24, 2016 5:39:30 PM org.apache.catalina.startup.VersionLoggerListener log
INFO: OS Name: Linux
Jul 24, 2016 5:39:30 PM org.apache.catalina.startup.VersionLoggerListener log
INFO: OS Version: 4.4.0-28-generic
Jul 24, 2016 5:39:30 PM org.apache.catalina.startup.VersionLoggerListener log
INFO: Architecture: amd64
Jul 24, 2016 5:39:30 PM org.apache.catalina.startup.VersionLoggerListener log
INFO: Java Home: /usr/lib/jvm/java-7-oracle/jre
Jul 24, 2016 5:39:30 PM org.apache.catalina.startup.VersionLoggerListener log
INFO: JVM Version: 1.7.0_80-b15
Jul 24, 2016 5:39:30 PM org.apache.catalina.startup.VersionLoggerListener log
INFO: JVM Vendor: Oracle Corporation
Jul 24, 2016 5:39:30 PM org.apache.catalina.startup.VersionLoggerListener log
INFO: CATALINA_BASE: /home/shravan/Projects/Webservices/.metadata/.plugins/org.eclipse.wst.server.core/tmp0
Jul 24, 2016 5:39:30 PM org.apache.catalina.startup.VersionLoggerListener log
INFO: CATALINA_HOME: /usr/share/tomcat7
Jul 24, 2016 5:39:30 PM org.apache.catalina.startup.VersionLoggerListener log
INFO: Command line argument: -Dcatalina.base=/home/shravan/Projects/Webservices/.metadata/.plugins/org.eclipse.wst.server.core/tmp0
Jul 24, 2016 5:39:30 PM org.apache.catalina.startup.VersionLoggerListener log
INFO: Command line argument: -Dcatalina.home=/usr/share/tomcat7
Jul 24, 2016 5:39:30 PM org.apache.catalina.startup.VersionLoggerListener log
INFO: Command line argument: -Dwtp.deploy=/home/shravan/Projects/Webservices/.metadata/.plugins/org.eclipse.wst.server.core/tmp0/wtpwebapps
Jul 24, 2016 5:39:30 PM org.apache.catalina.startup.VersionLoggerListener log
INFO: Command line argument: -Djava.endorsed.dirs=/usr/share/tomcat7/endorsed
Jul 24, 2016 5:39:30 PM org.apache.catalina.startup.VersionLoggerListener log
INFO: Command line argument: -Dfile.encoding=UTF-8
Jul 24, 2016 5:39:31 PM org.apache.coyote.AbstractProtocol init
INFO: Initializing ProtocolHandler ["http-bio-8080"]
Jul 24, 2016 5:39:31 PM org.apache.catalina.startup.Catalina load
INFO: Initialization processed in 781 ms
Jul 24, 2016 5:39:31 PM org.apache.catalina.core.StandardService startInternal
INFO: Starting service Catalina
Jul 24, 2016 5:39:31 PM org.apache.catalina.core.StandardEngine startInternal
INFO: Starting Servlet Engine: Apache Tomcat/7.0.68 (Ubuntu)
Jul 24, 2016 5:39:31 PM org.apache.catalina.startup.TldConfig execute
INFO: At least one JAR was scanned for TLDs yet contained no TLDs. Enable debug logging for this logger for a complete list of JARs that were scanned but no TLDs were found in them. Skipping unneeded JARs during scanning can improve startup time and JSP compilation time.
Jul 24, 2016 5:39:33 PM org.apache.catalina.startup.TldConfig execute
INFO: At least one JAR was scanned for TLDs yet contained no TLDs. Enable debug logging for this logger for a complete list of JARs that were scanned but no TLDs were found in them. Skipping unneeded JARs during scanning can improve startup time and JSP compilation time.
Jul 24, 2016 5:39:33 PM org.apache.catalina.core.ApplicationContext log
INFO: No Spring WebApplicationInitializer types detected on classpath
Jul 24, 2016 5:39:33 PM org.apache.catalina.core.ApplicationContext log
INFO: Initializing Spring root WebApplicationContext
Jul 24, 2016 5:39:33 PM org.springframework.web.context.ContextLoader initWebApplicationContext
INFO: Root WebApplicationContext: initialization started
Jul 24, 2016 5:39:33 PM org.springframework.context.support.AbstractApplicationContext prepareRefresh
INFO: Refreshing Root WebApplicationContext: startup date [Sun Jul 24 17:39:33 IST 2016]; root of context hierarchy
Jul 24, 2016 5:39:34 PM org.springframework.beans.factory.xml.XmlBeanDefinitionReader loadBeanDefinitions
INFO: Loading XML bean definitions from ServletContext resource [/WEB-INF/applicationContext.xml]
Jul 24, 2016 5:39:34 PM org.springframework.beans.factory.xml.XmlBeanDefinitionReader loadBeanDefinitions
INFO: Loading XML bean definitions from ServletContext resource [/WEB-INF/dataSource.xml]
Jul 24, 2016 5:39:34 PM org.springframework.web.context.ContextLoader initWebApplicationContext
SEVERE: Context initialization failed
java.lang.NoSuchMethodError: java.util.concurrent.ConcurrentHashMap.keySet()Ljava/util/concurrent/ConcurrentHashMap$KeySetView;
at org.apache.catalina.core.ApplicationContext.getInitParameterNames(ApplicationContext.java:368)
at org.apache.catalina.core.ApplicationContextFacade.getInitParameterNames(ApplicationContextFacade.java:367)
at org.springframework.web.context.support.WebApplicationContextUtils.registerEnvironmentBeans(WebApplicationContextUtils.java:200)
at org.springframework.web.context.support.AbstractRefreshableWebApplicationContext.postProcessBeanFactory(AbstractRefreshableWebApplicationContext.java:165)
at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:461)
at org.springframework.web.context.ContextLoader.configureAndRefreshWebApplicationContext(ContextLoader.java:381)
at org.springframework.web.context.ContextLoader.initWebApplicationContext(ContextLoader.java:293)
at org.springframework.web.context.ContextLoaderListener.contextInitialized(ContextLoaderListener.java:106)
at org.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:5068)
at org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5584)
at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:147)
at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1572)
at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1562)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Jul 24, 2016 5:39:34 PM org.apache.catalina.core.StandardContext listenerStart
SEVERE: Exception sending context initialized event to listener instance of class org.springframework.web.context.ContextLoaderListener
java.lang.NoSuchMethodError: java.util.concurrent.ConcurrentHashMap.keySet()Ljava/util/concurrent/ConcurrentHashMap$KeySetView;
at org.apache.catalina.core.ApplicationContext.getInitParameterNames(ApplicationContext.java:368)
at org.apache.catalina.core.ApplicationContextFacade.getInitParameterNames(ApplicationContextFacade.java:367)
at org.springframework.web.context.support.WebApplicationContextUtils.registerEnvironmentBeans(WebApplicationContextUtils.java:200)
at org.springframework.web.context.support.AbstractRefreshableWebApplicationContext.postProcessBeanFactory(AbstractRefreshableWebApplicationContext.java:165)
at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:461)
at org.springframework.web.context.ContextLoader.configureAndRefreshWebApplicationContext(ContextLoader.java:381)
at org.springframework.web.context.ContextLoader.initWebApplicationContext(ContextLoader.java:293)
at org.springframework.web.context.ContextLoaderListener.contextInitialized(ContextLoaderListener.java:106)
at org.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:5068)
at org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5584)
at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:147)
at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1572)
at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1562)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Jul 24, 2016 5:39:34 PM org.apache.catalina.core.StandardContext startInternal
SEVERE: One or more listeners failed to start. Full details will be found in the appropriate container log file
Jul 24, 2016 5:39:34 PM org.apache.catalina.core.StandardContext startInternal
SEVERE: Context [/service] startup failed due to previous errors
Jul 24, 2016 5:39:34 PM org.apache.catalina.core.ApplicationContext log
INFO: Closing Spring root WebApplicationContext
Jul 24, 2016 5:39:34 PM org.springframework.context.support.AbstractApplicationContext doClose
INFO: Closing Root WebApplicationContext: startup date [Sun Jul 24 17:39:33 IST 2016]; root of context hierarchy
Jul 24, 2016 5:39:34 PM org.springframework.context.support.AbstractApplicationContext doClose
WARNING: Exception thrown from ApplicationListener handling ContextClosedEvent
java.lang.IllegalStateException: ApplicationEventMulticaster not initialized - call 'refresh' before multicasting events via the context: Root WebApplicationContext: startup date [Sun Jul 24 17:39:33 IST 2016]; root of context hierarchy
at org.springframework.context.support.AbstractApplicationContext.getApplicationEventMulticaster(AbstractApplicationContext.java:346)
at org.springframework.context.support.AbstractApplicationContext.publishEvent(AbstractApplicationContext.java:333)
at org.springframework.context.support.AbstractApplicationContext.doClose(AbstractApplicationContext.java:880)
at org.springframework.context.support.AbstractApplicationContext.close(AbstractApplicationContext.java:841)
at org.springframework.web.context.ContextLoader.closeWebApplicationContext(ContextLoader.java:551)
at org.springframework.web.context.ContextLoaderListener.contextDestroyed(ContextLoaderListener.java:115)
at org.apache.catalina.core.StandardContext.listenerStop(StandardContext.java:5115)
at org.apache.catalina.core.StandardContext.stopInternal(StandardContext.java:5779)
at org.apache.catalina.util.LifecycleBase.stop(LifecycleBase.java:224)
at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:159)
at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1572)
at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1562)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Jul 24, 2016 5:39:34 PM org.springframework.context.support.AbstractApplicationContext doClose
WARNING: Exception thrown from LifecycleProcessor on context close
java.lang.IllegalStateException: LifecycleProcessor not initialized - call 'refresh' before invoking lifecycle methods via the context: Root WebApplicationContext: startup date [Sun Jul 24 17:39:33 IST 2016]; root of context hierarchy
at org.springframework.context.support.AbstractApplicationContext.getLifecycleProcessor(AbstractApplicationContext.java:359)
at org.springframework.context.support.AbstractApplicationContext.doClose(AbstractApplicationContext.java:888)
at org.springframework.context.support.AbstractApplicationContext.close(AbstractApplicationContext.java:841)
at org.springframework.web.context.ContextLoader.closeWebApplicationContext(ContextLoader.java:551)
at org.springframework.web.context.ContextLoaderListener.contextDestroyed(ContextLoaderListener.java:115)
at org.apache.catalina.core.StandardContext.listenerStop(StandardContext.java:5115)
at org.apache.catalina.core.StandardContext.stopInternal(StandardContext.java:5779)
at org.apache.catalina.util.LifecycleBase.stop(LifecycleBase.java:224)
at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:159)
at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1572)
at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1562)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Jul 24, 2016 5:39:34 PM org.apache.coyote.AbstractProtocol start
INFO: Starting ProtocolHandler ["http-bio-8080"]
Jul 24, 2016 5:39:34 PM org.apache.catalina.startup.Catalina start
INFO: Server startup in 3547 ms
Please point me out to the possible solution. Thank you so much.
I had the same problem recently.
It seems that Tomcat 7 classes were compiled using Java 8 libraries in Ubuntu 16.04, so you cannot run it using Java 7.
This is from catalina.jar (7.0.68) from repository:
# javap -verbose org/apache/catalina/core/ApplicationContext.class
Classfile /usr/share/tomcat7/lib/org/apache/catalina/core/ApplicationContext.class
Last modified 27/06/2016; size 33798 bytes
MD5 checksum b176f6c6f5854428fbfa67b298ed308a
Compiled from "ApplicationContext.java"
public class org.apache.catalina.core.ApplicationContext implements javax.servlet.ServletContext
SourceFile: "ApplicationContext.java"
InnerClasses:
public static #479= #478 of #943; //Dynamic=class javax/servlet/FilterRegistration$Dynamic of class javax/servlet/FilterRegistration
public static #479= #511 of #945; //Dynamic=class javax/servlet/ServletRegistration$Dynamic of class javax/servlet/ServletRegistration
public static #1005= #1004 of #2; //KeySetView=class java/util/concurrent/ConcurrentHashMap$KeySetView of class java/util/concurrent/ConcurrentHashMap
minor version: 0
major version: 50
flags: ACC_PUBLIC, ACC_SUPER
And this is from catalina.jar (7.0.68) downloaded from Apache:
# javap -verbose org/apache/catalina/core/ApplicationContext.class
Classfile /root/apache-tomcat-7.0.68/lib/org/apache/catalina/core/ApplicationContext.class
Last modified 08/02/2016; size 33585 bytes
MD5 checksum eda7f125c1ef506be98d360f6611a717
Compiled from "ApplicationContext.java"
public class org.apache.catalina.core.ApplicationContext implements javax.servlet.ServletContext
SourceFile: "ApplicationContext.java"
InnerClasses:
public static #481= #480 of #946; //Dynamic=class javax/servlet/FilterRegistration$Dynamic of class javax/servlet/FilterRegistration
public static #481= #513 of #948; //Dynamic=class javax/servlet/ServletRegistration$Dynamic of class javax/servlet/ServletRegistration
minor version: 0
major version: 50
flags: ACC_PUBLIC, ACC_SUPER
Both are compiled to class format 50 (Java 6), but the repository class has a link to a Java 8 exclusive method:
public static #1005= #1004 of #2; //KeySetView=class java/util/concurrent/ConcurrentHashMap$KeySetView of class java/util/concurrent/ConcurrentHashMap
So if you want to run Tomcat 7 with Java 7 in Ubuntu 16.04 you have to download it directly from Apache and install it manually.

Specs2 Specification Spring Autowire

I would like to use the Specs2 test framework for testing, but I am getting an java.lang.reflect.MalformedParameterizedTypeException when I try to integrate it with Spring.
There is a specs2-spring library, but that is using 2.10 and an old version of specs2. I looked at this question about scalaTest and it got me to this code:
#ContextConfiguration(Array("classpath:applicationContext.xml"))
class TryItTest extends SpecificationWithJUnit{
#Autowired val db:DriverManagerDataSource = null
new TestContextManager(this.getClass()).prepareTestInstance(this)
"this thing should" should{
"run ok" in {
db.getConnection
success
}
}
}
but when I try to run the test I get:
Nov 09, 2015 2:11:31 PM org.springframework.beans.factory.xml.XmlBeanDefinitionReader loadBeanDefinitions
INFO: Loading XML bean definitions from class path resource [applicationContext.xml]
Nov 09, 2015 2:11:31 PM org.springframework.context.support.GenericApplicationContext prepareRefresh
INFO: Refreshing org.springframework.context.support.GenericApplicationContext#243d315f: startup date [Mon Nov 09 14:11:31 EST 2015]; root of context hierarchy
Nov 09, 2015 2:11:31 PM org.springframework.jdbc.datasource.DriverManagerDataSource setDriverClassName
INFO: Loaded JDBC driver: oracle.jdbc.driver.OracleDriver
Nov 09, 2015 2:11:31 PM org.springframework.test.context.TestContextManager prepareTestInstance
SEVERE: Caught exception while allowing TestExecutionListener [org.springframework.test.context.support.DependencyInjectionTestExecutionListener#131746d9] to prepare test instance [my.package.TryItTest#12474a03]
java.lang.reflect.MalformedParameterizedTypeException
at sun.reflect.generics.reflectiveObjects.ParameterizedTypeImpl.validateConstructorArguments(ParameterizedTypeImpl.java:58)
at sun.reflect.generics.reflectiveObjects.ParameterizedTypeImpl.<init>(ParameterizedTypeImpl.java:51)
at sun.reflect.generics.reflectiveObjects.ParameterizedTypeImpl.make(ParameterizedTypeImpl.java:92)
at sun.reflect.generics.factory.CoreReflectionFactory.makeParameterizedType(CoreReflectionFactory.java:105)
at sun.reflect.generics.visitor.Reifier.visitClassTypeSignature(Reifier.java:140)
at sun.reflect.generics.tree.ClassTypeSignature.accept(ClassTypeSignature.java:49)
at sun.reflect.generics.repository.ConstructorRepository.getParameterTypes(ConstructorRepository.java:94)
at java.lang.reflect.Executable.getGenericParameterTypes(Executable.java:284)
at java.lang.reflect.Method.getGenericParameterTypes(Method.java:282)
at java.beans.FeatureDescriptor.getParameterTypes(FeatureDescriptor.java:387)
at java.beans.MethodDescriptor.setMethod(MethodDescriptor.java:116)
If I switch from SpecificationWithJUnit to SpecWithJUnit it runs the test without error, but I do not want to have in include all those traits by hand. I think (at least) one of the traits in Matchers is causing the problem. Has anyone else tried to do this with Specs2? Am I missing something important?
(I am using Spring V4.1.0.RELEASE)
Got it. I was not using the latest version of scalaz and specs2. Everything works perfectly now.

What's the difference between WebappClassLoader and StandardJarScanner?

I'm using the tomcat7-maven-plugin to launch a web application from Maven during the pre-integration-test lifecycle phase so that I can run tests against it locally during the integration-test phase.
I haven't worked much on the web application itself, but for the record it uses JSP and Java controllers which are tied together using Spring MVC 3.0.
When I run Maven (mvn clean verify) on my own machine I see the following output for the plugin:
[INFO] --- tomcat7-maven-plugin:2.2:run-war-only (start-tomcat) # MyProject ---
[INFO] Running war on http://localhost:8080/contentmain
[INFO] Using existing Tomcat server configuration at /Users/alexjohnson/MyWorkspace/MyProject/src/test/resources/com/MyCompany/MyProject/ui/automation/tomcat7
Jan 27, 2015 2:18:54 PM org.apache.catalina.core.AprLifecycleListener init
INFO: The APR based Apache Tomcat Native library which allows optimal performance in production environments was not found on the java.library.path: /Users/alexjohnson/Library/Java/Extensions:/Library/Java/Extensions:/Network/Library/Java/Extensions:/System/Library/Java/Extensions:/usr/lib/java:.
Jan 27, 2015 2:18:54 PM org.apache.coyote.AbstractProtocol init
INFO: Initializing ProtocolHandler ["http-bio-8080"]
Jan 27, 2015 2:18:54 PM org.apache.coyote.AbstractProtocol init
INFO: Initializing ProtocolHandler ["ajp-bio-8009"]
Jan 27, 2015 2:18:54 PM org.apache.coyote.AbstractProtocol init
INFO: Initializing ProtocolHandler ["http-bio-8443"]
Jan 27, 2015 2:18:54 PM org.apache.catalina.startup.Catalina load
INFO: Initialization processed in 425 ms
Jan 27, 2015 2:18:54 PM org.apache.catalina.core.StandardService startInternal
INFO: Starting service Catalina
Jan 27, 2015 2:18:54 PM org.apache.catalina.core.StandardEngine startInternal
INFO: Starting Servlet Engine: Apache Tomcat/7.0.53
Jan 27, 2015 2:18:55 PM org.apache.catalina.loader.WebappClassLoader validateJarFile
// More startup stuff
This is working as expected. Note that WebappClassLoader.validateJarFile is called. Also note that the APR info message is expected since we aren't using these libraries in the test environment.
When I run Maven on our build server (mvn clean verify) I see the following output for the plugin:
[INFO] --- tomcat7-maven-plugin:2.2:run-war-only (start-tomcat) # MyProject ---
[INFO] Running war on http://localhost:8080/contentmain
[INFO] Using existing Tomcat server configuration at /opt/jenkins/workspace/BuildServerWorkspace/src/test/resources/com/MyCompany/MyProject/ui/automation/tomcat7
Jan 27, 2015 2:30:09 PM org.apache.catalina.core.AprLifecycleListener init
INFO: The APR based Apache Tomcat Native library which allows optimal performance in production environments was not found on the java.library.path: /usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib
Jan 27, 2015 2:30:09 PM org.apache.coyote.AbstractProtocol init
INFO: Initializing ProtocolHandler ["http-bio-8080"]
Jan 27, 2015 2:30:09 PM org.apache.coyote.AbstractProtocol init
INFO: Initializing ProtocolHandler ["ajp-bio-8009"]
Jan 27, 2015 2:30:09 PM org.apache.coyote.AbstractProtocol init
INFO: Initializing ProtocolHandler ["http-bio-8443"]
Jan 27, 2015 2:30:09 PM org.apache.catalina.startup.Catalina load
INFO: Initialization processed in 823 ms
Jan 27, 2015 2:30:09 PM org.apache.catalina.core.StandardService startInternal
INFO: Starting service Catalina
Jan 27, 2015 2:30:09 PM org.apache.catalina.core.StandardEngine startInternal
INFO: Starting Servlet Engine: Apache Tomcat/7.0.53
Jan 27, 2015 2:30:10 PM org.apache.tomcat.util.scan.StandardJarScanner scan
WARNING: Failed to scan JAR [file:/opt/jenkins/BuildServerWorkspace/src/test/resources/com/MyCompany/MyProject/ui/automation/tomcat7/webapps/../../../../../../../../../../../target/MyProject/WEB-INF/lib/openws-1.5.1.jar] from WEB-INF/lib
java.io.FileNotFoundException: /opt/jenkins/BuildServerWorkspace/src/test/resources/com/MyCompany/MyProject/ui/automation/tomcat7/webapps/../../../../../../../../../../../target/MyProject/WEB-INF/lib/openws-1.5.1.jar (No such file or directory)
// Hundreds more FileNotFoundExceptions for JAR files.
This is not working. Note that this time StandardJarScanner.scan is called. Also note that the missing JAR files appear to have been created during the build on my local machine at the same relative path (11 directories out from webapps and 5 more into the target directory). I believe all of these JAR files are for third-party Maven dependencies.
I've read the official JavaDocs, but I can't find a helpful description of the WebappClassLoader or the StandardJarScanner. All I can really gather is that they're both used for loading classes from JAR files when Tomcat starts up.
What I'd like to know:
What are WebappClassLoader and StandardJarScanner specifically being used for in this context?
What are the key differences between WebappClassLoader and StandardJarScanner?
What sort of configuration could be different on the build server that causes it to use the StandardJarScanner instead of the WebappClassLoader when running Maven there (All of the environment variables and Maven parameters which I know of are already the same)?
have a look at this issue https://issues.apache.org/jira/browse/MTOMCAT-239
Sounds like some weird paths from jenkins, I can't figure out why..
WebAppClassLoader and StandardJarScanner are apples and oranges. StandardJarScanner always runs before WebAppClassLoader but only produces log statements if something goes wrong. This means that the StandardJarScanner log statements were appearing at the same point in the failing build as the WebAppClassLoader log statements in the passing build.
Since I was mis-reading these log statements I ended up asking the wrong question. My original problem still stands. I've opened a new question for it at Why is Tomcat throwing FileNotFoundExceptions for existing JAR files?.

SpringSource Tool Suite - can't run a Spring Template Project

I'm trying to learn about Spring using SpringSource Tool Suite. I haven't had much successful experience with either at this stage.
So far I've been able to install STS and create a new project, but haven't gotten much further. The problem seems to occur when running the application on the tc Server. I'll run through the steps I've taken and perhaps someone can point out where I'm going wrong. For the record, I'm using Ubuntu 11.10.
I installed STS using the this link, which I found on the
SpringSource Tool Suite Download page.
During installation of STS, I chose to install in my home folder
/home/luke/springsource. I left all of the packs selected:
SpringSource Tool Suite 2.8.0.RELEASE; vFabric tc Server
2.6.1.RELEASE; Spring Roo 1.1.5.RELEASE; Apache Maven 3.0.3
When selecting the JDK path, I chose /usr/lib/jvm/default-java, which
links to /usr/lib/jvm/java-6-openjdk.
The installation completes successfully and I launch STS manually by
going to the installation directory
/home/luke/springsource/sts-2.8.0.RELEASE and executing the file STS.
I had to do this to get STS to run despite having checked the box to
run STS now at the end of installation (not sure if that's a real
problem but thought it'd be worth mentioning).
I choose my workspace
/home/luke/Documents/workspace-sts-2.8.0.RELEASE and clicked OK. So
far so good?
In the dashboard I click Create > Spring Template Project.
In the following dialog I click Spring MVC Project and click Next,
then agree to download.
In the Project Settings dialog, I give the project a name, springmvc,
and a valid top-level package, then click Finish.
I can now see my newly created project in the Package Explorer, as
well as a Servers folder, plus the VMware vFabric tc Server Developer
Edition v2.6 in the Server window.
I drag the project to the server in the Server window and the project
appears there with a [Synchronized] status. I then right-click the
project in the Package Explorer and click Run As > Run on Server.
The aforementioned server is there and selected, so I click Next. The
project I created is in the Configured section of the next screen, so
I click Finish. I'm asked if I want to use Spring Insight, so I click
Yes and the server starts up.
Here's where the big trouble begins.
I get the following output to the console and a 404 message in the
browser window, which is pointed at http://localhost:8080/springmvc/
Console output:
Nov 3, 2011 4:21:29 PM com.springsource.tcserver.security.PropertyDecoder <init>
INFO: tc Runtime property decoder using memory-based key
Nov 3, 2011 4:21:30 PM com.springsource.tcserver.security.PropertyDecoder <init>
INFO: tcServer Runtime property decoder has been initialized in 924 ms
Nov 3, 2011 4:21:32 PM org.apache.coyote.AbstractProtocol init
INFO: Initializing ProtocolHandler ["http-bio-8080"]
Nov 3, 2011 4:21:32 PM com.springsource.tcserver.serviceability.rmi.JmxSocketListener init
INFO: Started up JMX registry on 127.0.0.1:6969 in 368 ms
Nov 3, 2011 4:21:32 PM org.apache.catalina.startup.Catalina load
INFO: Initialization processed in 3809 ms
Nov 3, 2011 4:21:32 PM org.apache.catalina.core.StandardService startInternal
INFO: Starting service Catalina
Nov 3, 2011 4:21:32 PM org.apache.catalina.core.StandardEngine startInternal
INFO: Starting Servlet Engine: VMware vFabric tc Runtime 2.6.1.RELEASE/7.0.20.B.RELEASE
Nov 3, 2011 4:21:32 PM org.apache.catalina.startup.HostConfig deployDescriptor
INFO: Deploying configuration descriptor springmvc.xml from /home/luke/springsource/vfabric-tc-server-developer-2.6.1.RELEASE/spring-insight-instance/conf/Catalina/localhost
Nov 3, 2011 4:21:33 PM org.apache.catalina.startup.SetContextPropertiesRule begin
WARNING: [SetContextPropertiesRule]{Context} Setting property 'source' to 'org.eclipse.jst.jee.server:springmvc' did not find a matching property.
Nov 3, 2011 4:21:33 PM com.springsource.insight.collection.tcserver.ltw.TomcatWeavingInsightClassLoader start
INFO: Context [localhost|springmvc] will not be woven
Nov 3, 2011 4:21:34 PM org.apache.catalina.core.StandardContext listenerStart
SEVERE: Error configuring application listener of class org.springframework.web.context.ContextLoaderListener
java.lang.NoClassDefFoundError: org/springframework/context/ApplicationContextException
at java.lang.Class.getDeclaredConstructors0(Native Method)
at java.lang.Class.privateGetDeclaredConstructors(Class.java:2406)
at java.lang.Class.getConstructor0(Class.java:2716)
at java.lang.Class.newInstance0(Class.java:343)
at java.lang.Class.newInstance(Class.java:325)
at org.apache.catalina.core.DefaultInstanceManager.newInstance(DefaultInstanceManager.java:119)
at org.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:4660)
at org.apache.catalina.core.StandardContext$1.call(StandardContext.java:5226)
at org.apache.catalina.core.StandardContext$1.call(StandardContext.java:5221)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:679)
Caused by: java.lang.ClassNotFoundException: org.springframework.context.ApplicationContextException
at org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1678)
at org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1523)
... 14 more
Nov 3, 2011 4:21:34 PM org.apache.catalina.core.StandardContext listenerStart
SEVERE: Skipped installing application listeners due to previous error(s)
Nov 3, 2011 4:21:34 PM org.apache.catalina.core.StandardContext startInternal
SEVERE: Error listenerStart
Nov 3, 2011 4:21:34 PM org.apache.catalina.util.SessionIdGenerator createSecureRandom
INFO: Creation of SecureRandom instance for session ID generation using [SHA1PRNG] took [227] milliseconds.
Nov 3, 2011 4:21:34 PM org.apache.catalina.core.StandardContext startInternal
SEVERE: Context [/springmvc] startup failed due to previous errors
Nov 3, 2011 4:21:34 PM org.apache.catalina.startup.HostConfig deployDescriptor
INFO: Deploying configuration descriptor insight.xml from /home/luke/springsource/vfabric-tc-server-developer-2.6.1.RELEASE/spring-insight-instance/conf/Catalina/localhost
Nov 3, 2011 4:21:41 PM com.springsource.insight.collection.tcserver.ltw.TomcatWeavingInsightClassLoader start
INFO: Context [localhost|insight] will not be woven
Nov 3, 2011 4:21:42 PM org.apache.catalina.core.ApplicationContext log
INFO: Initializing Spring root WebApplicationContext
Nov 3, 2011 4:22:00 PM org.apache.jasper.compiler.TldLocationsCache tldScanJar
INFO: At least one JAR was scanned for TLDs yet contained no TLDs. Enable debug logging for this logger for a complete list of JARs that were scanned but no TLDs were found in them. Skipping unneeded JARs during scanning can improve startup time and JSP compilation time.
Nov 3, 2011 4:22:11 PM org.apache.catalina.core.ApplicationContext log
INFO: Initializing Spring FrameworkServlet 'Spring MVC Dispatcher Servlet'
Nov 3, 2011 4:22:18 PM org.apache.catalina.startup.HostConfig deployDirectory
INFO: Deploying web application directory manager
Nov 3, 2011 4:22:18 PM com.springsource.insight.collection.tcserver.ltw.TomcatWeavingInsightClassLoader start
INFO: Context [localhost|manager] will not be woven
Nov 3, 2011 4:22:18 PM org.apache.catalina.startup.HostConfig deployDirectory
INFO: Deploying web application directory ROOT
Nov 3, 2011 4:22:18 PM com.springsource.insight.collection.tcserver.ltw.TomcatWeavingInsightClassLoader start
INFO: Context [localhost|ROOT] will not be woven
Nov 3, 2011 4:22:18 PM org.apache.coyote.AbstractProtocol start
INFO: Starting ProtocolHandler ["http-bio-8080"]
Nov 3, 2011 4:22:19 PM org.apache.catalina.startup.Catalina start
INFO: Server startup in 46286 ms
Any help in deciphering this, figuring out a solution and pointing out what I'm doing wrong here in what seems like a very simple use case of STS, would be greatly appreciated.
ditto Ralph.
selected the top level project, went to
Run As > Maven Clean
Run As > Maven Install
Run As > Run on Server
and it went through correctly. I'm not sure why it was failing on the first place though, or why installing it twice fixes it.
Looks like the Project is not correct deployed to the server.
The context menu, for the server contains a option to clean it. After you have clean it use the same context menu and publish the application.
Then try it again.
(An other cause for this problem can be that there is a compiler failure.)

Resources