Extreme Java

In my job, they officially use Eclipse Europa to develop. I really dislike Eclipse - it is a matter of taste. Because of this, I’m using NetBeans ability to share projects (like in this demo) - and it works like a charm.

But, since even rose has its thorn, they have some custom tasks here:

1. Inside every project, there’s a “config” folder with some files that must be copied to a “/var/xxx” folder before running the application;
2. A custom ant build script to create a ZIP file that will go to test and production environments; and
3. The file from step 2 must be transfered via SFTP to specific servers, on a folder named like “/var/deploy/xxx”.

These are manually-repeated tasks that must be done for every project. Eclipse users (here) must do it step-by-step. NetBeans users will use a (custom) brand-new module that I developed to do these tasks.

Just to enumerate, we need to solve these problems:

1. The “config” folder must be visible in “Projects” tab (it is under “Files” tab, but it is ugly);
2. An ant task in a custom build script must be called whenever I want;
3. The “config” folder must be copied to “/var/xxx” when the project is compiled;
4. An ZIP file must be sent via SFTP to a “/var/deploy/xxx” folder;
5. I don’t want to hardcode the “xxx” - what about a new tab on project properties?

OK. We have a lot of things to cover here. But, they are very simple. Since it could become a large post, I’ll cover them in other posts (that will be have a pingback here, on comments section).

I’m trying NetBeans 6.1 Beta. It is pretty stable to a beta, but I got a little problem: the “Find Usages” resource has some kind of memory leak. I sent the log using NetBeans’ built-in feature and voilá: it was sent successfully, and, like a charm, the server told me the issue was already sent by another user and is FIXED - yup, they are quick.

For the folks that laugh at Windows “send error” button, just take a look at the screenshot:

And, if you want to, you can see the report number 13304, and the issue #125531.

This one is one of the weirdest (and useful) things I ever seen in 16 years of my life as a developer (I’m 26, BTW)… OpenOffice is a very powerful Office Automation solution… NetBeans Plaform is a very powerful application platform… Can you imagine what you get when you mix both together? Yup, this is possible. With NB and OOo installed, you just need to create a NB Module with following JARs from OOo:

• juh.jar
• jurt.jar
• officebean.jar
• ridl.jar
• unoil.jar

I created another module with a single TopComponent, and add this snippet:

private OOoBean oo;

@Override
public void componentOpened() {
  add(oo = new OOoBean());
  oo.loadFromURL("private:factory/swriter", null);
  oo.aquireSystemWindow();
}

But, this will NOT work… If you try to run it, an exception will be thrown, because OOo libs (.DLL/.SO) will not be found. They only exists in OOo install folder, and are automagically found by OOo classes - only if they aren’t moved from the “classes” folder in OOo installation.

The workaround is a couple of classes in OOo SDK and The Ultimate Blackmagic(TM) - patent pending:

1. Grab odk / source / com / sun / star / lib / loader from OOo’s CVS and add it to integration module.
2. Add unowinreg.dll to Windows Path (if you are on Linux, skip this step).
3. We need to add “officebean” to classpath. Here lies OOoBean, and is optional if you are only using UNO interfaces. Other JARs are correctly listed by UnoInfo. Alter Loader.java and add this after line 205 (after “UnoInfo” call):

   vec.add(new File(fClassesDir, "officebean.jar").toURL());

4. Now, the ugliest part of the evil potion: create a Facade. Using OOoBean directly will lead to some ugly ClassLoader conflicts. I’ve no time to fix this, but the following works:

   public static Container buildOOoBean() {
     return (Container) Loader.getCustomLoader().
       loadClass("com.sun.star.comp.beans.OOoBean").
       newInstance();
   
   }
   public static void showOOoBean(Container oo) {
     Class c = oo.getClass().
       getClassLoader().
       loadClass("com.sun.star.beans.PropertyValue");
   
     Class ca = Array.newInstance(c, 0).getClass();
     Method m = oo.getClass().
      getMethod("loadFromURL", String.class, ca);
   
     m.invoke(oo, "private:factory/swriter", null);
     oo.getClass().
       getMethod("aquireSystemWindow").invoke(oo);
   }

5. Back to the TC, I use “buildOOoBean” in ComponentOpened and “showOOoBean” AFTER the component is shown (does not work otherwise).

This level of indirection is mandatory, because client module’s ClassLoader is different than integration module’s. If I do “OOoBean oo = new OOoBean()”, the declaration and instantiation will try to use the JARs bundled. Since they don’t lie on OOo install folder, the libraries will not be found. If I use “oo = (OOoBean) Loader.getCustomLoader().getClass(”…”).newInstance()”, the “OOoBean” class from declaration will NOT be the same as the class instantiated. If you ever worked with JavaEE 4 and JSF/Struts/etc, you know that same classes from different ClassLoaders throws ClassCastException.

Yes, it is the purest evilness, but works (with some bugs):

OOo works really well with Java, but NetBeans ClassLoader is “self-contained” (i.e. your bundled application must include EVERYTHING you use). This is a very nice architecture, but does not help if you if you need to interface with external applications.

5 minutes playing with the new NetBeans 6 platform and I found a very useful feature. In previous post, I download URL. Using that code, I want to create a list of POJOs (that will be represented by a list of Nodes). In old NB5 days, I had to create a Thread and a lot of syncronization code to make the download run in background. In NetBeans 6 is easy as 1, 2, 3:

1. Subclass ChildFactory (from Nodes API), implementing createKeys and createNodeFromKey
2. Create an AbstractNode passing Children.create(MyChildFactory)
3. Compile, run and enjoy!

It’s time to run the Profiler. I start JBoss with “run-nbprofiler.bat” (created by NetBeans) and ask NB to attach profiler. I’m using “analyze performance”, filtering Java core classes. JBoss runs very slow, but this is normal, since profiler is collecting execution of every method.

My first try showed that Lucene is called a lot of times, even when server is on “idle” state. Maybe it is a background job that should have a smaller frequency.

The second try filters out Lucene classes (using NB Profiler options). I found two bootlenecks: com.liferay.portal.lucene.IndexWriterFactory.decrement (takes 58% of startup time) and JBoss classloaders. And I found that Liferay auto deploy is running too often (~5 seconds of delay). Nice for develoment, awful for production use.

I had to remove JBoss classes from profiling, and I got amazed on things I discovered. I got a OutOfMemoryError (perm gen space) - solved by adding “-XX:MaxPermSize=256m” to run script - but I could find a lot of interesting things:

1. getResourceAsStream was called 12068 times;
2. Xerces’ ChildNode was instantiated 853808 times (does anybody imagine why Java is “slow”?);
3. Stacktrace got really deep. About 50 levels or more, not including JBoss, Lucene and JDK classes;
4. A chain of 7 filters was called before hitting Liferay’s MainServlet. If you consider a little forward made after MainServlet, we reach 14 filters before hitting a JSP file;
5. JSP compilation took 19s on this environment - total execution was 180s;
6. LR’s VelocityTaglib has 8 iconXXX() methods that took 8 seconds each - detail: each one forwards another request;
7. Everytime a “include” is made, the chain of 7 filter is called. And there’s a LOT of includes.

Remember that this was only ONE hit and the request wasn’t complete, because of the OOM error. After that, I’m going to have some fun in the “real world”. I’ll try to go deeper tomorrow. Maybe I can send a RFE to Liferay team after I organize the arguments.

Now, I have a profiled environment. But, when I tried to run JBoss (without profiling), I got an error in counter-ejb module. Its classpath is not correctand, so, I added this line to my build.[user|computer].properties:

classpath.manifest=[original line in build.properties]
                   lib/doubleclick-filter.jar

I reported this bug on Liferay JIRA [#LEP-2406]. After another 16 minutes of compilation, I forgot to initialize the database. I had two options: use a diferent connection pool or initialize it before deploying Liferay. I prefer the second, so, the easiest way was to create a MBean that depends on LiferayPool. Hypersonic is smart enough to allow multiple SQL commands in one Statement. I will upload the code later, but it is a matter of create a MBean that reads the script provided by Liferay and runs it on the poll.

This leads to an strange error about Spring transactions with EJB, Hibernate, JBoss and all. I don’t remember the details, but the solution is to change the debug attribute of CachedConnectionManager. This is something I can’t do in my shared JBoss server, so, I used an dirty trick:

1. Start JBoss with no Liferay, but with the original data source (liferay-db.xml configured);
2. JBoss translates your -ds file into and -service file and logs it with debug level;
3. Grab the translated file in log, create the -service, delete the -ds and indent the file - this will help you understand its structure;
4. Now, the funny part: copy CacheConnectionManager definition from jboss-jca.xml, paste into liferay-service.xml (inside the “CachedConnectionManager” optional attribute), and rename the MBean to an unique name - I put a “,name=xxx” suffix.

This will create a custom CCM to Liferay, without violating the original instance. That’s what I love in Java (specially in JBoss): you can create a Lego-like software that is just a matter of do the bindings.

Liferay runs fine, but I found two bugs: a ClassNotFoundException about ical4j, and the contents of Guest community are blank. The first, I solve by adding lib/ical4j.jar to manifest classpath (as above). The second I don’t care, since I will clean everything when I deploy the real application.

I got some weird exceptions with JBoss Portal, so, I decided to play around with Liferay. I have it running on my shared JBoss, and I have some ideas to their structured articles feature, so I will try harder on Liferay.

I’ve decided to check the bootlenecks on Liferay, because it is too slow. I’ll use NetBeans Profiler. Using NB’s ability to create projects using existing Ant scripts, I’ve done some setup:

1. Create an Java Application project using an already existing Ant script;
2. Add all “src” folder to the source folders list - about 20 of them (I guess this isn’t necessary unless we want to change anything);
3. Adjust compilation build to the “start” target and run to “deploy” (Liferay does not set “start” as dependency to “deploy”);
4. Create a “build.[user|computer].properties” and “app.server.[user|computer].properties” to customize some build parameters. I dislike Jikes, so, I’m using “javac.compiler=modern”. The rest of properties are straighforward to customize, but folders on app.server must be correctly configured - JBoss predefined values does not work on an out-of-box installation;
5. After building a lot of modules (about 10 minutes on an almost empty Windows box powered by an Athlon XP 3200+), more than 2000 classes - yes, two thousands - are compiled without errors. Running the “deploy” target will install Liferay on JBoss. It installs some JARs on server’s lib folder - I will change this later, before uploading to the real server.

Now, you can run JBoss. But first, some one-time configuration:

1. Add a datasource. To use profiling, I created a memory-only HSQLDB:

   <?xml version="1.0" encoding="UTF-8"?>
   <datasources>
    <local-tx-datasource>
     <jndi-name>jdbc/LiferayPool</jndi-name>
     <connection-url>jdbc:hsqldb:.</connection-url>
     <driver-class>org.hsqldb.jdbcDriver</driver-class>
     <user-name>sa</user-name>
     <password></password>
    </local-tx-datasource>
   </datasources>

2. Use NetBeans Profiler (Profile | Attach Profiler) to prepare a special run script to active profiling on JBoss - this file will be called “run-nbprofiler”.

Now, we have an working environment. Next step is test and profile.

Liferay is running slowly on my JBoss server, crashes lots of times, but works. I have decided to play with its source code, maybe optimizing here and there. NetBeans Profiler is perfect for this job, but Liferay was written with Eclipse. NetBeans project importer worked, but it didn’t help, because the Liferay source structure is monolithic. Since I had some free time this weekend, I’m manually converting the project. I converted about 15%, but look what I found so far:

• Tab size = 8. This is a HUGE tab size. Usually Java projects use 4. But this is a matter of taste.
• Actually, this is what I didn’t found: documentation. No JavaDocs on classes I opened. I don’t know how they can work on classes without any documentation. I always forget the use of any class I don’t open often…
• Lots of third part libraries. And I’m not talking about the classic ones (like commons-logging). I found things like EasyConf (looks like Java Preferences API), OSCache (object cache that works on cluster environments) and Trone (yet another Collections framework). This explains the need for a bigger PermSize.

Liferay is a great product, but it really needs a good cleanup. Some of their optimization tricks creates overheads, like using Colt (yet another Collections framework - yes, they have TWO different YACF). Colt have some “sync collections” that works using the same synchronization principle as Hashtable and Vector. The question is: why they leave synchronization to the collection? What about using the “synchronized(xxx)” structure?

I will follow GrOG’s tip (in last blog’s comments): report my discoveries to the community. I preferred to report only after deciding which Portal I will use, but the decision is taking me too deep on each implementation I’m testing.

Tired of lose the fight against NetBeans MDR stuff, I decided to check out Code Folding. Pretty easy, indeed. I haven’t done a full code fold example, but I was able to use NB ‘editor-fold’ pseudo-tag. Considering that you have the syntax support (see NBP Tutorials), the steps are:

1. The first step is not obvious, but it is the most important step. Without it, you will never activate folding (this was hard to find out - like any other “non-layer.xml” stuff). On your SettingsInitializer, you need to add this line:

   settingsMap.put(SettingsNames.CODE_FOLDING_ENABLE, Boolean.TRUE);

2. Next, we need to tell CustomFold which tokens it must use to find the pseudo-tag. On your EditorKit, you need to implement the SyntaxUpdateTokens class and put it as a document property. Like this:

   public class MyEditorKit extends NbEditorKit {
     [ ... ]        
   
     protected void initDocument(BaseDocument doc) {
       doc.putProperty(SyntaxUpdateTokens.class, new SUT());
     }
   
     public class SUT extends SyntaxUpdateTokens {
       private List list;
   
       public void syntaxUpdateStart() {
         list = new ArrayList();
       }
   
       public List syntaxUpdateEnd() {
         return list;
       }
   
       public void syntaxUpdateToken(TokenID tid, TokenContextPath tcp,
                                     int offset, int len) {
         if (tokenID == MyTokenContext.LINE_COMMENT) {
           list.add(new TokenInfo(tid, tcp, offset, len));
         }
       }
     }
   }

3. This is a weird step. I don’t know why there’s no ready-to-go class to do this job. You need to create a three-line class to add the sidebar to the editor:

   public class MyFoldingSideBarFactory implements SideBarFactory {
     public JComponent createSideBar(JTextComponent jTextComponent) {
       return new CodeFoldingSideBar(jTextComponent);
     }
   }

4. Now, glue everything in your layer.xml, under “Editor/MIME” folder:

   <folder name="SideBar">
     <file name="mypkg-MyFoldingSideBarFactory.instance"/>
     <attr name="org-netbeans-editor-GlyphGutter.instance
           /mypkg-MyFoldingSideBarFactory.instance"
           boolvalue="true"/>
   </folder>
   <folder name="FoldManager">
     <file name="org-netbeans-editor-CustomFoldManager$Factory.instance"/>
   </folder>

Small note: implementations of FoldManager can be chained, just like the SideBar. Making it easy to add a “real” code fold based on tokens.

This is the first post. I don’t have any ready tip yet, but I’ll post my “current mission”:

I’m trying to understand how to add a new language to NetBeans. I want everything Java Editor has: Code Completion, Code Folding, etc. In theory, it’s simple: I just need to add editor support and create a metamodel to feed other subsystems.

The first part is well explained on NetBeans Platform web site (actually, it’s incredibly simple to add syntax highlight), but there aren’t good docs about how the “Java Language Model” works on NB.

And now my problem: I’ve created a metadata repository using NetBeans’ MDR stuff, but I simply do not know how to populate the repository using source files (and how to create source files from the repository).