Jeff Mesnil

node.x, a Asynchronous Event Framework for the JVM ☛

Tim Fox:

What is Node.x?

• A general purpose framework that uses an asynchronous event based style for building highly scalable network or file system aware applications
• Runs on the JVM.
• Everything is asynchronous.
• Embraces the style of node.js and extends it to the JVM. Think node.js on steroids. Plus some.
• Polyglot. The same (or similar) API will be available in multiple languages: Ruby, Java, Groovy, (Python?, JS?, Clojure?), etc
• Goes with the recent developments with InvokeDynamic in Java 7 and bets on the JVM being the future premier runtime for dynamic languages.
• Enables you to create network servers or clients incredibly easily.
• True threading. Unlike node.js, Python Twisted or Ruby EventMachine, it has true multi-threaded scalability. No more spinning up 32 instances just to utilise the cores on your server.
• Understands multiple protocols out of the box including: TCP, SSL, UDP, HTTP, HTTPS, Websockets, AMQP, STOMP, Redis etc
• Provides an elegant api for composing asynchronous actions together. Glue together HTTP, AMQP, Redis or whatever in a few lines of code.

A framework a la node.js (and some more) in a multithreaded environment on the JVM is an intriguing idea with a lot of potential and the examples look both simple and powerful.

This project is based on Netty, an asynchronous event-driven network application framework, that I can’t stop to praise. We use it in HornetQ and is a big reason why HornetQ is so fast.

Tim is an ex-colleague from Red Hat and the previous lead of HornetQ. He knows his stuff about performance, concurrent and asynchronous code and I am intrigued to see where he will go with node.x (esp. if he adds a Clojure layer on top of it, functional programming is a good match for an asynchronous event framework).

Code Quarterly’s Q&A with Rich Hickey ☛

A very interesting interview of the creator of Clojure, Rich Hickey, by Michael Fogus, the co-author of Joy of Clojure¹.

If you appreciate this interview, Rich’s talk about Value, Identity and State is a must-see for anybody interested in computing and software languages (no knowledge of Lisp or Clojure is required).

1. It is on my reading list but I have not read it yet. ↩

Java API, Trade-Off Between Ugliness and Durability

In my previous post, I complained that JMX API was not consistent about the return types of DynamicMBean‘s getAttribute() and getAttributes() methods.

Eamonn McManus replied with a very good explanation on why getAttributes() is designed that way in order to be consistent with setAttributes(). This bit of “ugliness” in the API is a very good compromise. It requires some attention from the API user but it helps tremendously developers providing MBeans as they don’t have to deal with the atomicity of setting different attribute values. I would still object that getAttribute() should have been consistent and return an Attribute instead of the value. But I can understand that the Attribute indirection has no purpose in that case.

I like JMX API for that reason: it’s both simple and flexible. There are some parts which are ugly or require more work than I would like (writing an OpenMBean is a chore with too much information that should be handled by the JVM or JMX library) but I understand that JMX is widely used and backwards compatibility is a mandatory requirement. If every new release of Java was coming with a refined JMX API without a regard to backwards compatibility, I would not praise it like I do.

Writing an API both simple and empowering is hard. Support it over several releases is a tremendous task.In the Java world, Eclipse is the front leader on that task.

Several years ago, I developed a plug-in, eclipse-jmx, to manage JMX application inside Eclipse (it is also integrated into JBoss Tools as its JMX tool). And release after release, it is still supported by Eclipse without any changes to the API calls.

The last release of eclipse-jmx was written against Eclipse 3.2 and 3 years and 4 Eclipse releases later, eclipse-jmx still works fine, you can even install it using Eclipse Marketplace. I have a love/hate relationship with Eclipse as I use it everyday but I have only admiration for its developers who have been able to craft such a software and took on themselves to support APIs over long time and multiple versions instead of pushing all the work to the contributors to constantly update their plug-ins for each new release. The beauty of Eclipse APIs is in their durability and resiliency.

If you want to know how to write great Java API, the best resources to start are Eclipse and JMX.

On a similar topic, I am having a deep look at HTML5 and its various JavaScript APIs (WebSockets, geolocation, storage, canvas, etc.). I am curious to see how they will evolve and if lessons will have been learnt from the evolution (or lack of) of the DOM API…

Nice Clojure Outline in TextMate

+

TextMate is my favorite text editor and my preferred IDE when I’m not writing Java or Objective-C (for these two, I use Eclipse and Xcode).

I am currently learning Clojure and installed a bundle to have proper syntax highlighting and macros in TextMate.

I found a cool feature by accident yesterday. My file was starting to be “big” and I added comments to separate the different “sections”:

;; ************* ;;
;; Configuration ;;
;; ************* ;;
 
;; Configuration-related code
 
;; ********* ;;
;; DB Layer ;;
;; ********* ;;
 
;; DB-related code
 
;; *********** ;;
;; Web  Layer ;;
;; *********** ;;
 
;; Web-related code
 
;; ******* ;;
;; Startup ;;
;; ******* ;;
 
(db/init)
(server/start)

While I was writing code, I wanted to move from one part of the file to another and I used TextMate’s outline for that (the outline is the right part of the status bar at the bottom of the window, when you click it, it displays a list of all defined functions in the file). I noticed that these special comments were also displayed:

Clojure Outline in TextMate (click for full size)

I don’t know if the “magic” comes from TextMate or the Clojure bundle. To make it appear in the outline, the comment must end with ;;:

;; This will appear in TextMate outline ;;
 
;; This won't

I tried several comment formats in Ruby code but I was not able to make them appear in the outline. I assume the kudos should go the the bundle’s authors rather than to TextMate¹.

Outline the “logical” structure of the code

It is a very good idea to display these comments in the outline as they help to show the logical structure of a file.

For example in Java, the code is often organized as public/protected/private methods but it is does not give enough information on the logical composition of the class. I add comments to delimit the various interfaces implementations:

class Foo extends Thread implements IFoo, IBar {
 
    // IFoo implementation
    ...
    // IBar implementation
    ...
    // Thread overrides
    ...
    // Public
    ...
    // etc.
    ...
}

This valuable information is not displayed in Eclipse, in its outline or any other views². This valuable information is lost while it would greatly improve the outline and make it more useful.

I already encountered such “special comments” when I was developing an iPhone application with Xcode. It supports a special preprocessor directive to display text in the outline:

#pragma mark This will appear in Xcode outline
 
// This is a regular Objective-C comment
// it won't appear in the outline

I used them to split my Objective-C class in different logical sections (e.g. controller implementation, UI actions, protocol delegate):

Objective-C Outline in Xcode (click for full size)

I rarely uses code outline. Most of the time, I navigate in the code using the keyboard or the mouse wheel. Naturally, I prefer the outline to not clutter the UI when I do not use it.
However, when I want to use it, I prefer when it shows a “logical” outline of the code and not only a simple listing.

Compared to TexMate or Xcode, I find that Eclipse’s outline is cluttered:

Java Outline in Eclipse (click for full size)

I can not use it without either maximizing the view or scroll it horizontally (both requires too much UI gestures before reading the outline). I also find that it is overloaded with too much information:

• different icons for private/protected/public fields
• icon overlays for static, final modifiers or warnings
• nested hierarchy losing valuable UI space

All these details brings noise instead of information: an outline should display a general view of the class, not all these tiny details.

Using quite often these three softwares, I find TextMate and XCode approach to outlines more adapted to my workflow:

• they do not show it until you require it
• It displays a logical view of the code (using the comments/directive to show the logical sections)
• it does not show too much details not relevant to a broad view of the code

1. To be fair, TextMate deserves a meta-kudos to allow such cool features through its bundle mechanism! ↩
2. I have not check if NetBeans or IDEA can display “special comments” in their outlines but I never heard about such a thing… ↩

Concurrency Is My New Memory Management Criteria

Way back in school, I started studying object-oriented programming language with C++. I found the object concepts simple and powerful (especially after learning procedural programming with Fortran) but I did not like C++.

I browsed the internet to find if there were other OO languages that I could use instead of C++. I quickly found Java (1.0.x at the time, if I remember correctly) and preferred it for 3 reasons:

1. the concept and syntax were similar to C++ but simpler and well-defined. I could still leverage what I learn with C/C++ and applied it to Java
2. the documentation was good: with the javadoc, the whole platform API was one-click away
3. it had an automatic garbage collector

So I started to study Java seriously during my spare time, I found internships where I could learn from other Java developers and I ended up using it for most of my professional career.

Retrospectively, reasons #1 and #2 were nice to have but automatic garbage collection was the real decision criteria.

When I develop an application, most of the time I don’t care when I should allocate objects, if should I take ownership or release them, etc. I just want to have an object and use it¹.

But C++ forced me to think about all these details while Java made it simpler: new Foo() and that’s it. Back then, I was just starting to learn programming languages and I preferred to let the languages handle the memory: it would be less error-prone than if I had to do it manually.

My student friends told me that my Java programs would consume more memory than the C++ equivalent and likely run slower. Maybe it was true at the time, but the Java programs were simpler and faster to write, less buggy and less likely to leak memory during their lifetime.
So I chose Java and never looked back to C++.

Back to the present.

We now have multiple cores in our machines and to use them efficiently, we need to increase the concurrency of our code.

And I’m not enthusiastic with Java. The java.util.concurrent library helps a lot but I find really hard to write correct concurrent code and too easy to screw it up. I’m in the same state of mind that when C++ was “forcing” me to think about memory management.

I have added other languages to my toolbox (mainly Ruby/JRuby and Javascript). They complement nicely Java but they are not more fit for concurrency. There should be a language/platform that can handle concurrency for me or at least help me write correct concurrent code.

Which languages/platform will I use for that? Erlang, Clojure², Scala, Reia, other?
Which concepts should I use? STM, messages, actors, fork/join, Grand Central Dispatch, other?
I don’t even know if I am looking for a new platform or another language on top of the Java platform.

But I know that I am looking for something to help me with concurrency the same way Java helps me manage the memory.

Concurrency is likely to be one of the main decision criteria when I choose which next language/platform to learn. I envision it being as important as memory management was when I started to learn OO and how it tipped the balance for Java instead of C++.

Choosing Java over C++ shaped the beginning of my career. Choosing a concurrent language/platform may have the same importance for the next part of it…

1. I waited the release of Objective-C 2.0 to learn about Mac programming. I wanted to learn this language since I bought my first Powerbook but I waited until they added a garbage collector because I was not interested to manually manage the memory. Finally I still ended up having to learn it since the garbage collector is not available on the iPhone. Unsurprisingly, I have to track down a few memory leaks in the iPhone applications I develop. ↩
2. I received the printed version of Programming Clojure a few days ago. I like what I am reading so far. ↩