An awfull backend for AOP vs AspectWerkz (part 1)

(imported from http://blogs.codehaus.org/people/avasseur, read comments there)

Today while having my daily googling time on AOP, I found a 25 pages article about JMangler, a class load time weaving solution I had already bloged about.

The paper is here as a PDF.

I am rather surprised when I read that and mirror it with the fact that AOP would gain from massive adoption. I will defend my position in a very near future, but i could not avoid this flameware snip.

JMangler quotes itself as A Powerful Back-End for Aspect-Oriented Programming. And you might even buy a Prentice Hall book in 2004 to learn that.
I will save your time and money, by explaining you (again) why AOP does not gain from such a low concrete achievement strategy.

Just to let you think about by your own while I am writting my flameware:

• Will you buy a book in 2004 to learn that JMangler is used in AspectWerkz ? This is false: JMangler had been in AspectWerkz but has been kicked out since july 2003 and we are happy.


• Have your read that AspectWerkz hooking provides native support for BEA JRockit ?


• Have you already worked with IBM JRE class loading scheme


• Should a powerfull solution - that claim to be compliant with J2EE - fail under WebSphere environments ?
  
• Should a powerfull solution be SUN JRE 1.4 centric ?


• Should it hang when used in J2EE architectures ?


• Should it run only in -Xdebug mode, with two JVM side by side ?

Is that a joke ? Abusing the AOP hype in the Academic sphere ?
Software is a moving thing, and AspectWerkz makes AOP move fast. Be ready.

AOP debugging makes its show

(imported from http://blogs.codehaus.org/people/avasseur, read comments there)

Samuel has done a viewlet where he demonstrates how to easily debug an application and its aspects with AspectWerkz from within Eclipse.

If you did not yet understood how standard and easy it was from my first annoucement, you have to watch it now !

All is explained step by step. This is a definitely a things to work with.

AspectWerkz announces support for BEA JRockit

(imported from http://blogs.codehaus.org/people/avasseur, read comments there)

The latest snapshot of AspectWerkz is now supporting JRockit JVM for on the fly weaving.

By using JRockit JMAPI capabilities, AspectWerkz brings AOP one step ahead by officially supporting all major JVM (Sun, IBM, BEA). This allows to hook AspectWerkz in a seamless way to provide dynamic on the fly weaving of Aspects no matter your environment.

JRockit could not be supported before due to lack of HotSwap support and a limitation that forbids overriding of java.lang.ClassLoader with a -Xbootclasspath option. Users had to use offline mode until there.

The extension will be core part of coming 0.9 AspectWerkz release, so that all VM are supported with the smallest integration cost (Sun, IBM, JRockit).

Tests have been made with

• JRockit 7SP4 (java 1.3.1)


• JRockit 8.1SP1 (java 1.4.1_03)

The JMAPI JRockit API (management API) allows to add a classPreProcessor mechanism using a java snip or a command line option.

A single VM is thus running at full speed, both under java 1.3 and 1.4, without having to use bootclasspath override.

The classPreProcessor instance is responsible to modify bytecode of the class at load time. AspectWerkz was already providing this classPreProcessor based architecture since summer 2003 for Sun VM and Q3 2003 for IBM VM.

It is possible to hook or unhook AspectWerkz programmatically using JMAPI:

JVMFactory.getJVM().getClassLibrary().setClassPreProcessor(
   new JRockitPreProcessor());

To launch a JVM with AspectWerkz hooked in for class load time weaving this consists of a single option:

/jrockit/bin/java
-Xmanagement:class=org.codehaus.aspectwerkz.extension.jrockit.JRockitPreProcessor ....

If you are a JRockit fan, please help us and give a try to AspectWerkz online mode by your own. We need your feedback to make it suits your needs.

We will provide extensive documentation with the 0.9 release, but for the impatient :

• grab a CVS snapshot


• check bin/aspectwerk scripts to adapt them for JRockit (one line is commented out - notice that AspectWerkz needs to be in bootclasspath due to JMAPI)


• have a look at org.codehaus.aspectwerkz.extension.jrockit.JRockitPreProcessor in src/extensions/


• compile extensions.jar with ant/maven aspectwerkz:extensions:compile


• give it a try

ThreadLocal and memory leaks

(imported from http://blogs.codehaus.org/people/avasseur, read comments there)

Last day I realized that using ThreadLocal can be very dangerous when it comes to long running applications and garbage collections.

When you read the documentation of the ThreadLocal, you read that the object put in a ThreadLocal context is associated to the thread, and will be garbaged once the thread is dead.


Each thread holds an implicit reference to its copy of a ThreadLocal as long as the thread is alive and the ThreadLocal object is accessible; after a thread goes away, all of its copies of ThreadLocal variables are subject to garbage collection (unless other references to these copies exist).


So if you use ThreadLocal to store some object instance there is a high risk to have the object stored in the thread local never garbaged when your app runs inside an app server like WebLogic Server, which manage a pool of working thread - even when the class that created this ThreadLocal instance is garbage collected.

To solve this issue, you will probably have to check for it first with a memory profiler tool like JProfiler, and then wrap the value put in the ThreadLocal in a WeakReference.

That did not appears to be obvious while reading the javadoc. Isn't it ?

AspectWerkz online mode and JSR 163

(imported from http://blogs.codehaus.org/people/avasseur, read comments there)

Since end of july the JSR 163 draft is in public review. I did not had time to read it before (shame on me !) and Jonas has send me the link yesterday.
(read the draft)

I am pretty happy with it. As you might know, we have struggled hard in AspectWerkz to provide a way to weave class at load time and keep compliant with complex class loader hierarchies as J2EE ones.
If JMangler was good for first AspectWerkz releases (prior to 0.8), having our own architecture was a major step forward (and read recent post on JMangler lists that state problems under J2EE environment where it is reported to freeze - as I reported it in Q2 2003).

On july 30 I complained about JVMPI API and was kidding at Sun, since they had forgetted some information I needed in the API.
(old article)

I forgot to subscribe to JSPA to gain access to JSR for early reviewing. I should have (shame on me!)
This JSR 163 sounds really good.
- First I will have the information I need in the class load hook event.
- Second, there seems to have support for in process hotswapping
- Last and not least, there seems to have no requirements for -Xdebug for the hotswapping part.

So adopt AspectWerkz online architecture today and you will have a preview of what this JSR 163 is going to provide in a simpler way for java 1.5. I have already a todo list for AspectWerkz online architecture when this java 1.5 will be ready for prime time.

And don't forget: subscribe to JSPA. I have sent the agreement to SUN this sunday !

An old crap we don't care about anymore

(imported from http://blogs.codehaus.org/people/avasseur, read comments there)

With the 0.8 release and the new online mode architecture, a major limitation has been resolved. Now online mode (on the fly weaving) can be enabled with application servers (was released summer 2003).

Prior to 0.7.4 (including 0.7.4), it was not working correctly on some circumstances, due to a problem in the JMangler architecture which was used for online mode. You could use AspectWerkz offline mode as a fix to this.
Starting with 0.8, we have developped in AspectWerkz our own architecture, that fix the problem and brings much more functionnalities. AspectWerkz hook architecture is now used in CGLib as well since we made it available as a standalone module.

Some recent post on JMangler mailing lists reminded me this problem, and i want to give you some focus here about this topic, even if its an old story I don't care about anymore since the AspectWerkz way is now far from those limitations.



The problem was that JMangler is using a JPDA LaunchingConnector. This is a Sun provided JDI component that allow to launch a JVM in debug mode from a first JVM. The first JVM is responsible of hooking the low level layer in the second JVM when this one starts up. This interesting architecture provided by JMangler team has many problems.

The major one makes it bogus under heavy class loading scheme, where the application launched loads lots of class in a multi threaded way. This heavy class loading can appear when an application server starts. Lots of class are created and some execute thread queues are initialized, threads deploy applications, jdbc drivers are loaded ...

I did some test under a weblogic 7 startup sequence, and it just hangs. Hopefully AspectWerkz now works perfecly fine with the new 0.8 architecture.

If you are interested in reproducing the bug, which seems to be a JPDA bug that thus affects the whole JMangler 3.x architecture and thus AspectWerkz version prior to 0.8, you can use the following sample application (read the extended entry below).

Download the test application

Read more about AspectWerkz 0.8
Read more about the new on the fly weaving architecture



Description of the testing application:
A sample main class application starts T threads (configurable).
Each thread runs L loops (configurable).
At each loop the thread instantiates a new URLClassLoader with a search path on file system from which it loads a single class and instantiate a first instance. The class static init section (clinit) loads itself another class using another new URLClassLoader.

Each class loading is written on stdout: classname + hashcode. Hashcode allows to identify classes loaded by differents classloader which are not hierarchically linked.

This simulates a startup sequence where a big number of class is loaded by several threads ( 2 x T x L class). No class reference is kept so all are elligible to garbage collection.

Results
The app performs well with 3 threads and 2000 loops (thus 12 000 class loaded) (use a heap of 128m and a new of 32m, used -verbose:gc option). Turning on -Xdebug or -Runjdwp does not affect the correct behavior.

When started thru JMangler or used with a jpda LaunchingConnector - the component used in JMangler 3.x - it does not complete, and the launching VM garbage collection sometimes turns completely crazy making a lot of minor garbage collections at around 6000 class loaded, and the whole app always hangs.

Tests were run under win2000 SP3 and solaris, under different java 1.4 version, with the same results.

Explanation
I don't have explanations. AspectWerkz 0.8 uses a ListeningConnector coupled to an AttachingConnector instead, and it performs fine.
Now you can startup a complete application server with AspectWerkz hooked in in online mode with a minimal overhead.
And the 0.8 release comes with many more options on this important topic, that makes AspectWerkz on the fly weaving an easy and non intrusive operation.

Download the test application

Read more about AspectWerkz 0.8
Read more about the new on the fly weaving architecture

AspectWerkz has support for IBM

(imported from http://blogs.codehaus.org/people/avasseur, read comments there)

I managed to bring IBM jdk support to AspectWerkz for on the fly weaving.
Read more on project blog

IBM bundles a specific JRE that I did not planned when hacking the JVM hook architecture that allow on the fly weaving, so some of you reported JVM crash. Now it is fixed (in CVS HEAD) and for other exotic VM, an explicit error message should be reported.

If you have issues with it, please provide feedback. The IBM jre (at least 1.3.x) seems to have many issues as regards jars packaging. f.e I had to remove index.lst from meta-inf/ in two third parties jars for it to work...

debug your AOP with AspectWerkz

(imported from http://blogs.codehaus.org/people/avasseur, read comments there)

Using the latest development of AspectWerkz, and thus the recently released 0.8.1, debugging step by step your aspects is made easy.

Indeed we now have support for standard IDE debugging, so that bytecode weaving process can be followed step by step as regular java.

Then when your weaved application is running, you can set breakpoints on your advices and debug the app as usual, as plain java, as you are probably used to do it.
Though the IDE does not have access to the source of the weaved classes, you can set breakpoints in the original method affected by the pointcuts where aspects are bounded.

Here are some screenshot with IntelliJ.

---

Here is a short how-to:

Create a new project in your IDE, with AspectWerkz source distribution and your own sources.

Configure all classpath as needed.

On java 1.3, copy piccolo-xy.jar in your JAVA_HOME/jre/lib/ext/ (and remove it after since Ant does not like to have it there...)

Run or debug the class org.codehaus.aspectwerkz.hook.impl.WeavingClassLoader as usual with your IDE.
Use this class as a java command replacement.

Do not forget to define the -Daspectwerkz.definition.file

For example I am running it with the following to run a single unit test.
JVM arg = -Daspectwerkz.definition.file=src\test\aspectwerkz-test.xml -Daspectwerkz.transform.verbose=yes -Daspectwerkz.transform.dump=test..*


Program parameter = test.DynamicDeploymentTest


Working directory = my AspectWerkz home directory


AspectWerkz - new on the fly weaving architecture released

(imported from http://blogs.codehaus.org/people/avasseur, read comments there)

AspectWerkz 0.8 is released.

This version is a major change from the previous releases, since the whole on the fly weaving architecture has been reviewed.
There is now support for java 1.3, no debug mode required with certain options on java 1.4, and the most seamless way to plug the AspectWerkz AOP weaver in your environment thru a native module !


java -Xrunaspectwerkz [options] [classpath] my.Application

just hooks in the weaver for all the class loaded by the JVM (except bootstrap classloader).

Try it now

Read the official slides presenting the new architecture (pdf, ppt)

Read the documentation related to the new architecture

AspectWerkz online architecture - better than JVMPI - Sun needs to think about it for java 1.5

(imported from http://blogs.codehaus.org/people/avasseur, read comments there)

I already talked about the coming AspectWerkz new architecture for online mode (on the fly weaving). The key advantages will be as I explained:
- java 1.3 support
- debug mode for JVM not required
- option for single jvm
You may think its useless since the online weaving is already working fine (almost) with JMangler 3.x, which came with the original terrific idea of hotswapping the java.lang.ClassLoader, but I can bet you will change your position once you will have played with it a little more, thru the official 0.8 release.

Today I gave a look at the JVMPI API. Its a C layer receiving events from the JVM. Two events could be usefull to handle for AOP implementations: JVMPI_EVENT_CLASS_LOAD (sent when a class is loaded in the VM) and
JVMPI_EVENT_CLASS_LOAD_HOOK (sent when the VM obtains a class file data, but before it constructs the in-memory representation for that class).

Coding a layer receiving JVMPI_EVENT_CLASS_LOAD_HOOK to transform the bytecode could be usefull, since it could be enabled with just a -XrunMyStuff option and a shared library. This is the approach many commercial or open source debugging and profiling tools have choosen.

But giving a deeper look implementing an AOP layer on it will fail shortly for the following reasons:
- the event does not give the caller classloader information
- it might be a hell to call the java implemented AOP layer (AspectWerkz off course) from the C/C++ MyStuff JVMPI addon. Unless we setup a kind of client / server mode with a protocol to transmit bytecode back and forth (see AdaptJ for this sort of things).
Could lead to the idea of a distributed weaving engine... but do you need it ? And how will we be able to handle the calller classloader distinction, for example to weave struts classes in a specifc war only ?

So I definetely think the JMangler idea was good, even if the implementation is not that much good. And thus I am sure the new AspectWerkz architecture which unleash the idea will be and is already killing...

JVMPI doc
adaptJ project
JMangler project

As a ending note, I am really surprised Sun implemented the JVMPI_EVENT_CLASS_LOAD_HOOK in JVMPI and not directly (and even both) in the java.lang.ClassLoader... since its almost the single issue an on the fly weaver has to solve... since we have to override a class of rt.jar for java 1.3 compatibility... since BEA has implemented it since more than 2 years (except for the system classloader, just for the custom classloader deploying application like ear, war, ejb-jar, rar, but this might change with WebLogic 9)... since several project tries to address this lack (JMangler, beSee, AspectWerkz, JBoss, AspectJ)

Sun engineers - wake up - put that stuff in java 1.5 - java AOP needs it !
Ready to make a lobby with me ?

beSee donated to AspectWerkz

(imported from http://blogs.codehaus.org/people/avasseur, read comments there)

Past 6 month I have founded and developped the beSee project. This project is now closed, but I am definitely happy since the core architecture is now part of AspectWerkz, founded by Jonas Boner, for which I am commiter.

AspectWerkz power will be unleashed in the next weeks, and let me explain why.

It all started with beSee-1-x. This project aimed to provide a runtime bytecode instrumentation solution for BEA WebLogic Server (v7), based on Javassist for bytecode instrumentation and BEA ClassPreProcessor feature, which allow to modify bytecode of deployed applications when they are loaded in the app server.
beSee-1-x allowed easy instrumentation, based on a property file, and provided JMX management feature for hot configuration. It was easy to instrument on the fly some classes and run a load test on my app.

But there was something missing: BEA ClassPreProcessor does not allow instrumentation of core server classes or jvm wide component, like a JDBC driver, so I was unable to instrument enough my CMPs to track down SQL query being executed.

I started beSee-2-x architecture. It provides a JVM wide PreProcessor hook, allowing to extend what BEA provided for deployed application to the whole JVM, and thus to any product running java (!).
I knew JMangler add this kind of feature, but after some testing it appeared to be heavy and not that much stable as regards app server integration (let me know and I will send you some test case for it, and read more, you'll learn why).

So the main features of beSee-2-x I implemented are:
- plugable JVM wide ClassPreProcessor mechanism for on the fly instrumentation
- bytecode kit independancy (BCEL and Javassist provided, any other supported)
- offline compiler (the AspectJ way)
- performance wise:
- beSee-2-x does not requires JVM to run in debug mode (JMangler does)
- beSee-2-x is java 1.3 compatible (JMangler is not)
- beSee-2-x auto adapts behavior for java 1.3 and 1.4 (process forking or hotswapping)
- beSee-2-x allow to disable dual process, thus a standard java ... command to launch the target application to instrument on the fly is enough - no need for a wrapper script and stream piping in background (required when hotswap or process forking is used).

beSee-2-x allowed me to unleash the power of beSee-1-x to the whole weblogic, not just the deployed app.
And crucial point, its open architecture was ready to unleash power and capabilities of next generation AOP framework - and AspectWerkz is the one !

So stay tune. All those features are right now in AspectWerkz CVS, and in some weeks, you won't feel the pain to plug AOP in your whole architecture, on the fly or offline, and with its full power - no debug mode required, java 1.3 supported, optional process forking (choose between tuning and easy set-up), not tight to any app server but certified app server ready.

BCEL patch submitted

(imported from http://blogs.codehaus.org/people/avasseur)

I have submitted a patch for the BCEL thread safe issue I described (read).

You'll find it on the BCEL dev mailing list archive. I am waiting for BCEL guy to put the patch in their CVS HEAD. For now you can whether incorporate the patch in the BCEL source and rebuild your own BCEL, or build a jar with only the two files I patched and put it in your classpath before the bcel.jar.

(read the post on the list archive)

(dowload the patch for BCEL 5.1)

BCEL internal

(imported from http://blogs.codehaus.org/people/avasseur)

Using BCEL to do bytecode modification on a simple application I have, I encountered problems with thread safety. As soon as I got 2 threads instrumenting in parallel, the java Verifier begins to complains about generated method signatures.

I struggled half a day to find it out. Thread safety problem is not that easy to isolate. After some googling I found the following post on BCEL mailing lists

It appears that some of BCEL static class (org.apache.bcel.generic.Type and org.apache.bcel.classfile.Utiity at least), which should provides static method to compute human understandable signature from bytecode and vice-versa are miscoded.

If I understand that doing a thread safe application can be difficult, I am really thinking BCEL guy were lazy this time with a snip like

// sample purpose - pseudo code

package org.apache.bcel.classfile;

public abstract class Utility {

public static int consumed_chars;

public static String[] methodSignatureArgumentTypes(String) {
loop {
     // read and set consumed_chars field
     consumed_chars++;
}
}

public static String methodSignatureToString(String) {
loop {
    // read and set consumed_chars field
    consumed_chars++;
}
}

}

As we can see its obvious it is non thread safe because its bad designed: two static methods are using the same static field as a temporary storage to perform string operations.

I had to wrap the consumed_chars in a ThreadLocal and replace all calls. It works fine, but I cannot be sure I covered all the bad design BCEL put in.

Guy, ok to say BCEL is not designed for thread safety in the FAQ. But please try to avoid such ugly coding.

Another option for me could be to add some synchronized in my instrumentation application - but it would be a hell from a performance point of view.
Considering BCEL is used in app server modules (npulse, willy technology), it should be thread safe. In app server there is lots of classloading ocuring concurenlty (stub / skel generation, CMP impl at deployment time ...)