I've got an interesting question about classloader behavior.
Question one: What is the order in which the classloader will load jars?
The following jars and containing classes are given:
a.jar
+-com/scheffield/foo/A.class
b.jar
+-com/scheffield/foo/B.class
Which class will be loaded?
Question two: Is it true that the path and name of a file in the classpath is unique?
The following jars and containing classes are given (realworld example):
spring-beans-3.0.3.RELEASE.jar
+-META-INF/spring.schemas
spring-aop-3.0.3.RELEASE.jar
+-META-INF/spring.schemas
What I can tell you is that both files are be loaded by Spring otherwise an exception would occur (see this article).
Why am I asking that:
I made a so called big jar (cookbook entry for gradle). Thats a jar with the application classes and all other dependencies unzipped and packet in the big jar. And I'm not absolutely sure what to do with duplicated files.
Classes are resolved however a classloader wants to resolve them (that's the whole point of having a classloader architecture). Most classloaders you deal with in practice are variants of java.net.URLClassLoader which loads classes (and resources) based on a search path (class path) of directories and jars. Each location in the search path is treated as a source of classes and the locations are searched in order.
No, names are not unique. The first one encountered in the search order will be used.
If you combine jars into one big jar there is a definite possibility of conflict. If you are careful to merge them from the last source in your effective classpath to the first (thus overriding later jars with earlier jars), you will get approximately the same result.
I say approximately because the manifests in jars contain additional processing instructions that need to be merged as well. For example, a manifest can contain a Class-Path attribute that includes additional jars in the classpath. It's possible to merge jars but lose manifest attributes that are specifying part of your actual necessary classpath. If your manifest contains sealed or signed jars then you might not be able to do this merging at all without violating the signed parts of the jar.
In summary, jars are not really designed to be merged in this way. It can work but there are many possibilities for error, some of which are not possible to solve. One common cause of error is to merge two jar files and end up with more than one entry with the same path, which is allowed in zip files. The ant jar and zip tasks allow you to merge multiple sources and can produce these kinds of issues.
Really, it's better to instead bundle web apps consisting of many jars and sources into a single WAR or EAR archive. That's kind of the whole point of why they exist.
the files are being loaded in the order the containing jars appear on the classpath. this applies to classes. if you are loading other resources (like the spring.schema) you may use either classloader.getResource(...) or classloader.getResources(...). the first one returnes the first resource on the classpath, the second one also returnes shadowed resources.
I don't think a valid zip archive contains duplicate entries.
regards
Related
I am trying to use the Aspectj compiler ajc in a modular (OSGi setting). The standard way ajc seems to be used is to take aspects & java code and turn it into one a JAR with all classes and resources in the -inpath, -aspectpath, and -sourceroots.
I am trying to weave aspects an OSGi executable JAR from bnd. This executable jar contains a set of bundles that need to be woven. However, in a modular system, the boundary is quite important. For one, the manifest often contains highly relevant information to that bundle or one of the many extenders. Flattening all the classes into a big blog won't work.
I am therefore weaving each bundle separately. However, then the output is cluttered with the aspects. I'd like to import these to keep the aspect modules proper modules. However, using the annotation programming model, I notice that ajc is modifying the aspect modules, so I need to rewrite those as well. This is fine, but since I weave each bundle separately, I have the question if the weaving of the aspect could depend on what gets other modules woven? That is,
does the modification of the annotated aspect depend on the classes that it is woven in?
The other issue is what happens to resources with the same name? Since my -inpath is only one JAR (the bundle), I notice I end up with the correct manifest (META-INF/MANIFEST.MF) in the output. However, if the -inpath consists of many bundles, what will the manifest be? Or any other resource that has the same path and thus overlaps?
Last issue is external dependencies. I understand acj wants to see the whole world and include this whole world into the output JAR. However, I must exclude external dependencies of a bundle. Is there a way to mark JARs as: use, but do not include. A bit like the maven 'provided' scope?
Summary:
Does the modification of an #Aspect annotated class depend on the targets that is applied to?
Can I compile the #Aspect annotated classes into separate JARs?
How to handle the external dependencies that will be provided in the runtime and thus must be excluded from the output JAR.
What are the rules around overlapping resource paths in the -inpath and -sourceroots?
UPDATE In the mean time I've made an implementation in Bndtools.
Does the modification of an #Aspect annotated class depend on the targets that is applied to?
If you want to be 100% sure you have to read the AspectJ source code, but I would assume that an aspect's byte code is independent of its target classes, because otherwise you could not compile aspects separately and also not build aspect libraries.
Can I compile the #Aspect annotated classes into separate JARs?
Absolutely, see above.
How to handle the external dependencies that will be provided in the runtime and thus must be excluded from the output JAR.
If I understand the question correctly, you probably want to put them on the class path during compilation, not on the inpath.
What are the rules around overlapping resource paths in the -inpath and -sourceroots?
Again, probably you have to look at the source code. If I was you I would simply assume that the selection order is undefined and make sure to not have duplicates in the first place. There should be Maven plugins helping you with filtering the way you want the result to be.
bndtools seems to have close ties to Eclipse. So does AspectJ as an Eclipse project. Maybe you can connect with Andy Clement, the AspectJ maintainer. He is so swamped with his day-time job though, he hardly ever has any free cycles. I am trying to unburden him as much as I can, but OSGi is one of my blind spots and I hardly know the AspectJ source code. I am rather an advanced user.
In my maven project, there were two different jars but contain the same class with the same package name. Can that cause NoSuchMethodError?
I know if the method does't exist will cause this Error? What if there have the method, but duplicate?
This usually happens when you have two versions of the same project on the classpath and they changed the API. Depending on which JAR is looked at first, the method will be found or not.
If the method exists in both JARs, then it will work but something else might fail (there will be other differences).
If you add the same JAR several times on the classpath, then it will always work.
Solutions:
Always make sure the classpath is clean. Every fully qualified class name should resolve to a single class resource (file).
Sometimes, people add classes from different projects to their JARs "to make them easier to use." If that is the case, then open the JAR with a ZIP tool and delete the extra classes. Maven works better without such "help".
If the conflict exists, and the method exists in both classes, the error won't show, however its a common case that there's a difference in a signature, between the conflicting versions, which can confuse you. My advice, check if the signature of the loaded class matches, and work your exclusions properly
Maybe it is a silly question, but what is the best way to test if a Java Archive (jar) file is an OSGi bundle? That is, what are the minimal requirements for the jar to be fully compatible? Is it the mere presence of a META-INF/MANIFEST.MF (I don't think so)? If not, what are the minimal fields that must be provided by this file?
Practically, how should I test if the jar is an OSGi jar?
Look for the Bundle-SymbolicName header in the MANIFEST.MF. This is the only mandatory header in an OSGi bundle, at least since Release 4.0 of the OSGi specification. Therefore if the Bundle-SymbolicName header is defined, then the JAR is an OSGi bundle. If not, then it is just a JAR.
A bundle is a group of Java classes and additional resources equipped with a detailed manifest MANIFEST.MF file on all its contents, as well as additional services needed to give the included group of Java classes more sophisticated behaviors, to the extent of deeming the entire aggregate a component.
The OSGi spec describes the bundle as a "unit of modularization" that "is comprised of Java classes and other resources which together can provide functions to end users.".
a bundle is a JAR file that:
Contains [...] resources
Contains a manifest file describing the contents of the JAR file and providing information about the bundle
Can contain optional documentation in the OSGI-OPT directory of the JAR file or one of its sub-directories
In short, a bundle = jar + OSGI information (specified in the JAR manifest file - META-INF/MANIFEST.MF), no extra files or predefined folder layout are required.
I guess the only qualification be required is that there are a bundle of classes and that it contains a MANIFEST.MF file which contains valid OSGi headers.
Consider this link
As well as this
Answering your question, the only way to test the bundle it to check if the MANIFEST.MF file exists and contains valid headers.
When running the packaged app like "java -jar my-app-0.0.1-SNAPSHOT-jar-with-dependencies.jar", I get the following error:
org.springframework.beans.factory.xml.XmlBeanDefinitionStoreException: Line 47 in
XML document from URL [jar:file:/.../cxf/javafirst/target/my-app-0.0.1-SNAPSHOT-jar-with-dependencies.jar!/application-context.xml]
is invalid; nested exception is org.xml.sax.SAXParseException; lineNumber: 47;
columnNumber: 61; cvc-complex-type.3.2.2: Attribute 'sendServerVersion'
is not allowed to appear in element 'httpj:engine'.
This is due to an outdated http-jetty.xsd schema published at apache.org. And that is NOT my problem. My problem is that this does not happen in cases like:
maven jetty:run
or
maven exec:java
where the service runs directly against the unpacked binary directories and obviously finds the updated and correct http-jetty.xsd schema file.
I want to do basically one thing:
put the webservice into one jar including dependencies. That should also include all XML schema files because, obviously, it is not a good idea to make a web service dependent on outside resources.
What's the best way to do that using maven?
My possibly naive solution would be to copy the XML schema files manually into the resources directory and tell CXF to resolve them in the jar file. Therefore I have two more specific questions:
1.) Is it possible to let maven find the XML schema files and copy them into appropriate places?
2.) What's the recommended way to make CXF look up the schema files in the jar file?
3.) Is there any better, best-practice solution to that problem?
My maven configuration regarding the maven assembly plugin is the direct combination of the last two sections at http://maven.apache.org/plugins/maven-assembly-plugin/usage.html.
Spring has a very good mechanism to resolve the schema files that it requires - it typically does not download it from the web at all, instead using locally available files within jar files to get the schema and validate the xml, for eg. consider the context custom namespace schema in Spring, if you look at the META-INF/spring.schemas file in spring-context.jar file, you will see an entry along these lines:
http\://www.springframework.org/schema/context/spring-context-3.1.xsd=org/springframework/context/config/spring-context-3.1.xsd
basically what it is saying is to resolve the spring-context-3.1.xsd files from the classpath org.springframework.context.config.spring-context-3.1.xsd file
This applies for any third party library also, which in your case is http-jetty.xsd.
I think what I would recommend is to simply create a spring.schemas file in your jar file in META-INF/ folder, put an entry for the full path to the schema and replace it with a classpath version of http-jetty.xsd.
I have some questions derived from a problem that I have already solved through this other question. However, I am still wondering about the root cause. My questions are as follows:
What is the purpose of spring.handlers and spring.schemas?
As I understand it's a way of telling the Spring Framework where to locate the xsd so that everything is wired and loaded correctly. But...
Under what circumstances should I have those two files under the META-INF folder?
In my other question linked above, does anybody know why I had to add the maven-shade-plugin to create those two files (based on all my dependencies) under META-INF? In other words, what was the ROOT CAUSE that made me have to use the maven shade plugin?
What is the purpose of spring.handlers and spring.schemas?
well you more or less found it out by yourself, let's add some more details:
some spring libraries contain a spring.schemas and a spring.handlers file inside a META-INF directory
META-INF/spring.schemas
re-maps(*) schemalocation to a xsd inside the library
(abstract) only re-mapped versions are supported by this library
META-INF/spring.handlers
provides namespace handler classes for specific namespaces
the namespace handler class provides the parser logic to parse spring-batch beans, like job,
step, etc.
(*) the actual re-mapping happens during the build of the spring application context
Under what circumstances should I have those two files under the
META-INF folder?
normally the files are inside the spring library jars you use, but you can use the mechanism to implement own namespace bean parsing, then you would have own files
In my other question linked above, does anybody know why I had to add
the maven-shade-plugin to create those two files (based on all my
dependencies) under META-INF? In other words, what was the ROOT CAUSE
that made me have to use the maven shade plugin?
if you use a spring namespace in your spring configuration, you need the appropriate files
the problem arises when you want to run a java application:
with a main class either
the spring libraries need to be on the classpath
or all is merged into one jar, which has to be on the classpath (*)
as war/ear server application, the spring libaries need to be on the classpath, normally inside the war
i guess you did not start the mainclass with the complete classpath and i updated my answer for your first question too
(*) if you merge all into one jar, you have to make sure, that the contents of all spring.schemas/spring.handlers files are merged into one spring.schemas and one spring.handlers file, see this answer for a configuration with maven to create an all-in-one.jar