Trouble with class loading webapp JARs with Jetty 6 - jersey

Question

I have a Jetty 6.1.9 environment which I am trying to bootstrap an 'exploded' WAR style context, however, I am getting NoClassDefFoundError errors.
The Jetty env is bootstrapped like so:
public class MyServer {
public static void main(String[] argv) {
jetty_default = jetty_default + "/web-content/jetty-6.1.9";
String jetty_home = System.getProperty("jetty.home", jetty_default);
Server server = new Server();
Connector connector = new SelectChannelConnector();
connector.setPort(7895);
server.setConnectors(new Connector[]{connector});
WebAppContext myAppCxt= new WebAppContext();
myAppCxt.setContextPath("/foo");
myAppCxt.setWar(jetty_home + "/wardirs/myApp");
myAppCxt.setDefaultsDescriptor(jetty_home + "/etc/webdefault.xml");
myAppCxt.setInitParams(initParamsMaps);
myAppCxt.setTempDirectory(aTmpDir);
server.setHandler(processingNodeAppCxt);
server.start();
}
The layout of wardirs/myApp looks like so
wardirs
myApp
WEB-INF
lib\
jdom.jar
web.xml
The class I am getting the error for is in foo.jar.
It also feels like Jetty is finding the JAR in the WEB-INF/lib dir because after starting the server foo.jar becomes locked (I tried deleting it) and when I kill the server it becomes unlocked.
If I manually put the JARs on the classpath of the java cmd line which runs MyServer's main method then all works fine, however, I want these JARs to live in the WEB-INF\lib dir and be automatically picked up as they should be.
The actual exception is:
Mar 06, 2013 9:07:25 PM sun.reflect.NativeMethodAccessorImpl invoke0
SEVERE: /foo/api/process
java.lang.NoClassDefFoundError: org/jdom/Content
at com.foo.MyManager$Status.createICProcessor(MyManager.java:231)
at com.foo.myApp.Api.process(API.java:xxx)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$TypeOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:149)
at com.sun.jersey.server.impl.model.method.dispatch.ResourceJavaMethodDispatcher.dispatch(ResourceJavaMethodDispatcher.java:67)
at com.sun.jersey.server.impl.uri.rules.HttpMethodRule.accept(HttpMethodRule.java:259)
at com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:133)
at com.sun.jersey.server.impl.uri.rules.ResourceClassRule.accept(ResourceClassRule.java:83)
at com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:133)
at com.sun.jersey.server.impl.uri.rules.RootResourceClassesRule.accept(RootResourceClassesRule.java:71)
at com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:990)
at com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:941)
at com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:932)
at com.sun.jersey.spi.container.servlet.WebComponent.service(WebComponent.java:384)
at com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:451)
at com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:632)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:487)
at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:362)
at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:181)
at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:726)
at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:405)
at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
at org.mortbay.jetty.Server.handle(Server.java:324)
at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:505)
at org.mortbay.jetty.HttpConnection$RequestHandler.content(HttpConnection.java:842)
at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:648)
at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:211)
at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:380)
at org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:395)
at org.mortbay.thread.BoundedThreadPool$PoolThread.run(BoundedThreadPool.java:450)
Caused by: java.lang.ClassNotFoundException: org.jdom.Content
at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
at java.lang.ClassLoader.loadClass(ClassLoader.java:423)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
at java.lang.ClassLoader.loadClass(ClassLoader.java:356)
... 35 more
The only other thing I can think to mention since this feels like a classloader context issue is that the MyManager referenced in the top line of the stack trace is running in its own thread which was started by the webapp's ServletContextListener like so:
myMgrThread = new Thread(myManager);
long myMgrThreadId = myMgrThread.getId();
myMgrThread .setName("MyApp_MyMgr-" + myMgrThreadId);
myMgrThread .setPriority(Thread.NORM_PRIORITY);
myMgrThread .setDaemon(true);
myMgrThread .start();

Answer 1

According to your project layout, and that stacktrace, it found com.foo.myApp.Api and com.foo.MyManager in your WEB-INF/lib/foo.jar just fine, but it failed to find org.jdom.Content
Is org.jdom.Content present in your wEB-INF/lib/foo.jar? or in a similar file you haven't mentioned like WEB-INF/lib/jdom.jar? Or are you expecting to find org.jdom.Content from the parent classloader (in other words, whatever classloader the MyServer class was started from)?

Answer 2

I think I answered my own question. The IDE project this is running in does not move the 'webapp' .class files into "$jetty_home/wardirs/myApp/WEB-INF/classes" as one would expect per the servlet spec. Instead it includes them in the class path which invokes MyServer.main.
When I copied the application classes into "$jetty_home/wardirs/myApp/WEB-INF/classes" the NOCLASSDEFERRORs went away. So now the classes are WEB-INF/classes and on the classpath calling main.
As you might be able to tell here this project has no separation of concerns between the webapp and the container. All the classes and JARS for the application and jetty have been dumped together and it 'works'. I've been trying to teases things apart so that the webapp classes and dependent JARs are clearly separated, eg. introducing a ServletContextListener to move webapp code into which was previously being bootstrapped from 'main', moving JARs into WEB-INF/lib and the like. It's like trying to get chewing gum out of your hair.