The project I'm working on is considerably large. While trying to get load time weaving working for this spring project, I was instructed to use both the spring-instrument javaagent as well as the aspectjweaver javaagent. However, I notice that when using the aspectjweaver agent, my launch time shoots up 4-6 fold. I can also see identical weave messages 4-6 times coming from ContextOverridingClassLoader.
If I remove aspectjweaver however, and only use spring-instrument, I notice my launch time decrease dramatically with only a single weave message per join point coming from AppClassLoader.
The only issue being that some specific classes are not woven (I found that this is due to the spring application context not yet being loaded before the faulty classes are loaded by the class loader, as spring is the mechanism that enables the weaving). I've found a solution of my own by creating a custom javaagent which enables weaving in the same manner that spring-instrument does, only it does so in the premain rather than on application context load. It now weaves all the classes and in reasonable time.
However, I'd prefer not to go down this hacky road as I can only presume that the two agents were designed the way it is for a reason.
I wonder if anyone else has seen a similar issue with the aspectjweaver javaagent and if someone might know why that agent is so slow compared to just using spring-instrument.
If the answer interests anyone, I've figured out the issue.
Spring uses a temporary classloader ContextOverridingClassLoader to get metadata about the bean classes prior to actually loading them into the context.
The spring-instrument javaagent (or more accurately, the spring framework code which may or may not use the spring-instrument javaagent) specifically only weaves classes loaded by the classloader used to load the application context.
Code inside of InstrumentationLoadTimeWeaver$FilteringClassFileTransformer:
if (!this.targetClassLoader.equals(loader)) {
return null;
}
return this.targetTransformer.transform(
loader, className, classBeingRedefined, protectionDomain, classfileBuffer);
On the other hand, aspectjweaver does not have such a filtering mechanism and so will weave even those classes loaded by spring's temporary ContextOverridingClassLoader. Fortunately, aspectjweaver has an essentially undocumented system property (or at least I was unable to find any documentation on this) called aj.weaving.loadersToSkip. By setting this to:
-Daj.weaving.loadersToSkip=org.springframework.context.support.ContextTypeMatchClassLoader$ContextOverridingClassLoader
I was able to skip weaving for that classloader and speed up the loading of my application context tremendously.
Incidentally, I've found that both the spring-instrument and aspectjweaver ultimately both use ClassPreProcessorAgentAdapter to weave the classes, and thus it is probably not necessary to use both agents (aspectjweaver will weave a superset of the classes that spring-instrument will). However, depending on your configuration, the application might complain about the missing agent at startup so you might as well include it (at the cost of some additional unnecessary overhead).
Related
Reading the Spring AOP documentation (link), I'm having a hard time (maybe also because english is not my native language) understanding these paragraphs.
First, I read
Further, in certain environments, this support enables load-time
weaving without making any modifications to the application server’s
launch script that is needed to add
-javaagent:path/to/aspectjweaver.jar or (as we describe later in
this section)
-javaagent:path/to/org.springframework.instrument-{version}.jar
(previously named spring-agent.jar).
And
Developers modify one or more files that form the application context
to enable load-time weaving
Which files? #Aspect classes and aop.xml files?
Then, when describing an example in the same sub-chapter, they say
We have one last thing to do. The introduction to this section did say
that one could switch on LTW selectively on a per-ClassLoader basis
with Spring, and this is true. However, for this example, we use a
Java agent (supplied with Spring) to switch on the LTW. We use the
following command to run the Main class shown earlier:
And they apply a Java Agent to the JVM.
-javaagent:C:/projects/foo/lib/global/spring-instrument.jar
Now I have a couple of doubts.
If I #EnableLoadTimeWeaving, do I need the spring-instrument Jar file as Java Agent?
I suppose the answer is yes, because we need to add bytecode to the class file before loading it. But a confirmation would be much appreciated.
The Jar naming is a little ambiguos, first they mention spring-agent.jar, then they use org.springframework.instrument-{version}.jar, and then spring-instrument.jar.
Are we always talking about the same Jar file?
I see from another question you asked that you are using Spring Boot and running a fat jar. In this case you don't need #EnableLoadTimeWeaving or spring-instrument (formerly known as spring-agent). Just ignore them if you are not running in an appserver for which you don't control the agent path.
I opened an issue for you about the confusion in the docs: https://github.com/spring-projects/spring-framework/issues/22429.
We are working on a new project using Spring, Hibernate and Transaction management using AspectJ (#Transactional annotations) and we are not sure what is the best practices weaving options to go.
We've started with the LTW and placed the tomcat Loader, it worked fine.. but then we thought that it may be less risky to have compile time weaving which instruments and *.classes and not on load time (in memory), so it will done just one time and not when on the tomcat startup. This is done via Maven aspectj-maven-plugin plugin.
Can you please advice ?
What are the pros and cons of using those weaving options ?
Thanks!
Disclaimer: This is not a discussion forum but a Q/A platform, so your question does not have the correct answer, it rather sparks discussion. I am trying to elaborate a bit anyway.
You basically already stated the main facts about LTW versus CTW. I am failing to see why LTW should be more risky than CTW, though. It slows down your server start-up, but the risk is the same as with CTW because the resulting byte code is also the same. Or are you talking about the risk that maybe more code gets (e.g. 3rd party libraries) woven than you intend? In that case yet, CTW keeps you on the safe side and you have more control over what should be woven. It should also speed up your server start-up time in comparison with LTW.
So if you have no compelling reasons to use LTW, such as the wish to dynamically intercept code you are unaware of during development, go ahead and use CTW and you are on the safe side, as long as you are fine with adjusting your build process accordingly. AspectJ Maven plugin is pretty straightforward to use, so that should not be a big deal. You can still switch to LTW as needed and when needed. OTOH, if you are using LTW now and are fine with the server start-up time, maybe there is no need to switch. If unsure, try both approaches and compare the results. ;-)
My project has a set of custom defined annotations that could be present in any bundle deployed in the OSGi 4.3 framework. I want to find any class with these annotations in the classpath. I tried using BundleWiring.listResources(...) and Bundle.loadClass(...) for each class found. I have done some tests with an small set of bundles and it needs almost 200MB of Permanent Generation JVM memory space because all classes are loaded.
Is there a way to free loaded classes PermGen memory space when the program realizes that they does not have these annotations?
Is there a better way to look for annotated classes in an OSGi framework?
I think you should not do annotation scanning as it slows down startup and needs a lot of memory. JEE application servers do annotation scanning at startup to make lazy programmers happy and the result is very annoying (e.g. scan for JPA or EJB annotations).
I guess you are implementing a technology where you can define the rules. I suggest that you should define rules that are similar to these:
Annotate your class
Have a MANIFEST header where the annotated class must be listed.
An even better solution can be to use a custom capability namespace with specified attributes. E.g.:
Provide-Capability: myNamespace;classes=com.foo.myClass1,com.foo.myClass2
In your technology, you should write a BundleTracker that calls:
BundleWiring.getCapabilities("myNamespace");
If the namespace is present, you can find the classes that should be processed.
If you implemented the technology, you can consider an extension to Bnd to fill that MANIFEST header automatically. That extension can be used than when bnd is started from the command line or from build tools like maven.
Btw.: You can use ASM to parse the class bytecode or use the built in possibility of Java to build up AST. Although those could work to solve the memory issue, I still think that you should define the list of classes directly in the MANIFEST header as it makes things much more clear. You can read the MANIFEST headers, you can check the capabilities on webconsole but you cannot do the same with bytecode.
Usually, classpath scanning for annotations is a bad idea in an OSGi context, as the classpath is more like a graph. However, there are situations where this can be useful. Hence, OSGi encourages the usage of the Whiteboard Pattern.
What you could possibly do is register each of these classes as services in the OSGi registry. Then, create a separate bundle that just tracks these services and transforms/manipulates them in some way. For example, this project scans for all classes annotated with #Path and #Provider annotations, and transforms them into Jersey REST APIs.
I have a large spring project, using xml configuration. I'm looking for a quick way to verify changes to the xml configuration.
I can load the whole project locally - the problem is this takes more than 5 minutes, loads a huge amount of data.
My XML editor catches XML formatting errors.
I'm looking for something intermediate - to catch obvious problems like references to beans that aren't defined, or calling constructors with the wrong arguments. Is there a quick way to do this, without having to actually invoke all the constructors and bring up the whole environment?
I'm building with Maven and editing with Eclipse, although my question isn't specific to either.
Since you already use Eclipse, you could try Spring Tool Suite (comes either standalone or as an add-on). It's essentially Eclipse with extra Spring-specific features, like Beans Validator. I'm not sure how thorough the validation is, but it should catch most configuration problems.
It's maintained by SpringSource so its integration with Spring "just works" and it's guaranteed not be more or less in sync with Spring Framework's release cycle.
Beanoh :
http://beanoh.org/overview.html#Verify
this project does exactly what I'm looking for. Verify obvious problems with spring config, but without the overhead of initializing everything.
You can use a Spring testing support to integration test your Spring configuration. However if the loading of the context is taking 5 mins, then the tests will also take the same amount of time. Spring does cache the context so if you have multiple tests using the same set of Spring contexts, then once cached the tests should be very quick.
I can suggest a few ways to more efficiently test your configuration:
Organize your project in modules, with each module being responsible for its own Spring configuration - this way, each module can be independently developed and tested.
If you have a modular structure, the testing can be more localized by mocking out the dependent modules, again this is for speed.
I've seen this term when read about how Spring works and I've just read the article about JPA implementation performance and it has the next statistics:
EclipseLink 3215 ms
(Run-time weaver - Spring ReflectiveLoadTimeWeaver weaver )
EclipseLink (Build-time weaving) 3571 ms
EclipseLink (No weaving) 3996 ms
So, could someone explain in plain English, what is weaving?
Thanks!
Weaving is generating or editing code by directly modifying existing .class (byte-code) files.
This can occur at different points in the application life cycle.
Outside of JVM
at compile time
at packaging time
Inside a JVM
at class load time.
after a class has been loaded.
Spring Framework uses this for AOP functionality. Eclipselink uses weaving for lazy loading or change tracking.
From here:
In Spring AOP makes it possible to modularize and separate logging, transaction like services and apply them declaratively to the components Hence programmer can focus on specific concerns. Aspects are wired into objects in the spring XML file in the way as JavaBean. This process is known as 'Weaving'.
In nutshell, we could say
Weaving is the process of applying the Advices to the Target objects
at given pointcuts to get the Proxy Objects.
I found this description useful:
Weaving: This is the process of inserting aspects into the application code at the
appropriate point. For compile-time AOP solutions, this weaving is generally done
at build time. Likewise, for runtime AOP solutions, the weaving process is executed
dynamically at runtime [using JDK dynamic proxy and CGLIB proxy]. AspectJ supports another weaving mechanism called load-
time weaving (LTW), in which it intercepts the underlying JVM class loader and
provides weaving to the bytecode when it is being loaded by the class loader.
reference: Pro Spring 5: An In-Depth Guide to the Spring Framework and Its Tools
Object-oriented software systems that are developed
using aspect-oriented programming techniques
consist of classes and aspects. Classes implement
the primary functionality of an application,
for example, managing stocks or calculating
insurance rates. Aspects, on the other hand, capture
technical concerns like persistence, failure handling,
communication, or process synchronization.
There are two ways in which classes and aspects
can be woven: static or dynamic.
Static weaving means to modify the source code of a class by inserting aspect-speci c statements at
join points.In other
words: aspect code is inlined into classes. The
result is highly optimized woven code, whose execution
speed is comparable to that of code written
without using aspects.
Weaving is a technique of manipulating the byte-code of compiled Java classes.
Ref: http://www.eclipse.org/eclipselink/documentation/2.5/concepts/app_dev007.htm
Cheers!
Weaving is the process of linking aspect with other application types or objects to create an advised object. Weaving can be done at compile time, load time or runtime. Spring AOP performs weaving at runtime.