Adopting New JVM Languages in the Enterprise (Update) 102

Posted by Dean Wampler Thu, 15 Jan 2009 07:40:00 GMT

(Updated to add Groovy, which I should have mentioned the first time. Also mentioned Django under Python.)

This is an exciting time to be a Java programmer. The pace of innovation for the Java language is slowing down, in part due to concerns that the language is growing too big and in part due to economic difficulties at Sun, which means there are fewer developers assigned to Java. However, the real crown jewel of the Java ecosystem, the JVM, has become an attractive platform for new languages. These languages give us exciting new opportunities for growth, while preserving our prior investment in code and deployment infrastructure.

This post emphasizes practical issues of evaluating and picking new JVM languages for an established Java-based enterprise.

The Interwebs are full of technical comparisons between Java and the different languages, e.g., why language X fixes Java’s perceived issue Y. I won’t rehash those arguments here, but I will describe some language features, as needed.

A similar “polyglot” trend is happening on the .NET platform.

The New JVM Languages

I’ll limit my discussion to these representative (and best known) alternative languages for the JVM.

  1. JRuby – Ruby running on the JVM.
  2. Scala – A hybrid object-oriented and functional language that runs on .NET as well as the JVM. (Disclaimer: I’m co-writing a book on Scala for O’Reilly.)
  3. Clojure – A Lisp dialect.

I picked these languages because they seem to be the most likely candidates for most enterprises considering a new JVM language, although some of the languages listed below could make that claim.

There are other deserving languages besides these three, but I don’t have the time to do them justice. Hopefully, you can generalize the subsequent discussion for these other languages.

  1. Groovy – A dynamically-typed language designed specifically for interoperability with Java. It will appeal to teams that want a dynamically-typed language that is closer to Java than Ruby. With Grails, you have a combination that’s comparable to Ruby on Rails.
  2. Jython – The first non-Java language ported to the JVM, started by Jim Hugunin in 1997. Most of my remarks about JRuby are applicable to Jython. Django is the Python analog of Rails. If your Java shop already has a lot of Python, consider Jython.
  3. Fan – A hybrid object-oriented and functional language that runs on .NET, too. It has a lot of similarities to Scala, like a scripting-language feel.
  4. Ioke – (pronounced “eye-oh-key”) An innovative language developed by Ola Bini and inspired by Io and Lisp. This is the newest language discussed here. Hence, it has a small following, but a lot of potential. The Io/Lisp-flavored syntax will be more challenging to average Java developers than Scala, JRuby, Jython, Fan, and JavaScript.
  5. JavaScript, e.g., Rhino – Much maligned and misunderstood (e.g., due to buggy and inconsistent browser implementations), JavaScript continues to gain converts as an alternative scripting language for Java applications. It is the default scripting language supported by the JDK 6 scripting interface.
  6. Fortress – A language designed as a replacement for high-performance FORTRAN for industrial and academic “number crunching”. This one will interest scientists and engineers…

Note: Like a lot of people, I use the term scripting language to refer to languages with a lightweight syntax, usually dynamically typed. The name reflects their convenience for “scripting”, but that quality is sometimes seen as pejorative; they aren’t seen as “serious” languages. I reject this view.

To learn more about what people are doing on the JVM today (with some guest .NET presentations), a good place to start is the recent JVM Language Summit.

Criteria For Evaluating New JVM Languages

I’ll frame the discussion around a few criteria you should consider when evaluating language choices. I’ll then discuss how each of the languages address those criteria. Since we’re restricting ourselves to JVM languages, I assume that each language compiles to valid byte code, so code in the new language and code written in Java can call each other, at least at some level. The “some level” part will be one criterion. Substitute X for the language you are considering.

  1. Interoperability: How easily can X code invoke Java code and vice versa? Specifically:
    1. Create objects (i.e., call new Foo(...)).
    2. Call methods on an object.
    3. Call static methods on a class.
    4. Extend a class.
    5. Implement an interface.
  2. Object Model: How different is the object model of X compared to Java’s object model? (This is somewhat tied to the previous point.)
  3. New “Ideas”: Does X support newer programming trends:
    1. Functional Programming.
    2. Metaprogramming.
    3. Easier approaches to writing robust concurrent applications.
    4. Easier support for processing XML, SQL queries, etc.
    5. Support internal DSL creation.
    6. Easier presentation-tier development of web and thick-client UI’s.
  4. Stability: How stable is the language, in terms of:
    1. Lack of Bugs.
    2. Stability of the language’s syntax, semantics, and library API’s. (All the languages can call Java API’s.)
  5. Performance: How does code written in X perform?
  6. Adoption: Is X easy to learn and use?
  7. Tool Support: What about editors, IDE’s, code coverage, etc.
  8. Deployment: How are apps and libraries written in X deployed?
    1. Do I have to modify my existing infrastructure, management, etc.?

The Interoperability point affects ease of adoption and use with a legacy Java code base. The Object Model and Adoption points address the barrier to adoption from the learning point of view. The New “Ideas” point asks what each language brings to development that is not available in Java (or poorly supported) and is seen as valuable to the developer. Finally, Stability, Performance, and Deployment address very practical issues that a candidate production language must address.

Comparing the Languages

JRuby

JRuby is the most popular alternative JVM langauge, driven largely by interest in Ruby and Ruby on Rails.

Interoperability

Ruby’s object model is a little different than Java’s, but JRuby provides straightforward coding idioms that make it easy to call Java from JRuby. Calling JRuby from Java requires the JSR 223 scripting interface or a similar approach, unless JRuby is used to compile the Ruby code to byte code first. In that case, shortcuts are possible, which are well documented.

Object Model

Ruby’s object model is a little different than Java’s. Ruby support mixin-style modules, which behave like interfaces with implementations. So, the Ruby object model needs to be learned, but it is straightforward or the Java developer.

New Ideas

JRuby brings closures to the JVM, a much desired feature that probably won’t be added in the forthcoming Java 7. Using closures, Ruby supports a number of functional-style iterative operations, like mapping, filtering, and reducing/folding. However, Ruby does not fully support functional programming.

Ruby uses dynamic-typing instead of static-typing, which it exploits to provide extensive and powerful metaprogramming facilities.

Ruby doesn’t offer any specific enhancements over Java for safe, robust concurrent programming.

Ruby API’s make XML processing and database access relatively easy. Ruby on Rails is legendary for improving the productivity of web developers and similar benefits are available for thick-client developers using other libraries.

Ruby is also one of the best languages for defining “internal” DSL’s, which are used to great affect in Rails (e.g., ActiveRecord).

Stability

JRuby and Ruby are very stable and are widely used in production. JRuby is believed to be the best performing Ruby platform.

The Ruby syntax and API are undergoing some significant changes in the current 1.9.X release, but migration is not a major challenge.

Performance

JRuby is believed to be the best performing Ruby platform. While it is a topic of hot debate, Ruby and most dynamically-typed languages have higher runtime overhead compared to statically-typed languages. Also, the JVM has some known performance issues for dynamically-typed languages, some of which will be fixed in JDK 7.

As always, enterprises should profile code written in their languages of choice to pick the best one for each particular task.

Adoption

Ruby is very easy to learn, although effective use of advanced techniques like metaprogramming require some time to master. JRuby-specific idioms are also easy to master and are well documented.

Tool Support

Ruby is experiencing tremendous growth in tool support. IDE support still lags support for Java, but IntelliJ, NetBeans, and Eclipse are working on Ruby support. JRuby users can exploit many Java tools.

Code analysis tools and testing tools (TDD and BDD styles) are now better than Java’s.

Deployment

JRuby applications, even Ruby on Rails applications, can be deployed as jars or wars, requiring no modifications to an existing java-based infrastructure. Teams use this approach to minimize the “friction” of adopting Ruby, while also getting the performance benefits of the JVM.

Because JRuby code is byte code at runtime, it can be managed with JMX, etc.

Scala

Scala is a statically-typed language that supports an improved object model (with a full mixin mechanism called traits; similar to Ruby modules) and full support for functional programming, following a design goal of the inventor of Scala, Martin Odersky, that these two paradigms can be integrated, despite some surface incompatibilities. Odersky was involved in the design of Java generics (through earlier research languages) and he wrote the original version of the current javac. The name is a contraction of “scalable language”, but the first “a” is pronounced like “ah”, not long as in the word “hay”.

The syntax looks like a cross between Ruby (method definitions start with the def keyword) and Java (e.g., curly braces). Type inferencing and other syntactic conventions significantly reduce the “cluuter”, such as the number of explicit type declarations (“annotations”) compared to Java. Scala syntax is very succinct, sometimes even more so than Ruby! For more on Scala, see also my previous blog postings, part 1, part 2, part 3, and this related post on traits vs. aspects.

Interoperability

Scala’s has the most seamless interoperability with Java of any of the languages discussed here. This is due in part to Scala’s static typing and “closed” classes (as opposed to Ruby’s “open” classes). It is trivial to import and use Java classes, implement interfaces, etc.

Direct API calls from Java to Scala are also supported. The developer needs to know how the names of Scala methods are encoding in byte code. For example, Scala methods can have “operator” names, like ”+”. In the byte code, that name will be ”$plus”.

Object Model

Scala’s object model extends Java’s model with traits, which support flexble mixin composition. Traits behave like interfaces with implementations. The Scala object model provides other sophisticated features for building “scalable applications”.

New Ideas

Scala brings full support for functional programming to the JVM, including first-class function and closures. Other aspects of functional programming, like immutable variables and side-effect free functions, are encouraged by the language, but not mandated, as Scala is not a pure functional language. (Functional programming is very effective strategy for writing tread-safe programs, etc.) Scala’s Actor library is a port of Erlang’s Actor library, a message-based concurrency approach.

In my view, the Actor model is the best general-purpose approach to concurrency. There are times when multi-threaded code is needed for performance, but not for most concurrent applications. (Note: there are Actor libraries for Java, e.g., Kilim.)

Scala has very good support for building internal DSL’s, although it is not quite as good as Ruby’s features for this purpose. It has a combinator parser library that makes external DSL creation comparatively easy. Scala also offers some innovative API’s for XML processing and Swing development.

Stability

Scala is over 5 years old and it is very stable. The API and syntax continue to evolve, but no major, disruptive changes are expected. In fact, the structure of the language is such that almost all changes occur in libraries, not the language grammar.

There are some well-known production deployments, such as back-end services at twitter.

Performance

Scala provides comparable performance to Java, since it is very close “structurally” to Java code at the byte-code level, given the static typing and “closed” classes. Hence, Scala can exploit JVM optimizations that aren’t available to dynamically-typed languages.

However, Scala will also benefit from planned improvements to support dynamically-typed languages, such as tail-call optimizations (which Scala current does in the compiler.) Hence, Scala probably has marginally better performance than JRuby, in general. If true, Scala may be more appealing than JRuby as a general-purpose, systems language, where performance is critical.

Adoption

Scala is harder to learn and master than JRuby, because it is a more comprehensive language. It not only supports a sophisticated object model, but it also supports functional programming, type inferencing, etc. In my view, the extra effort will be rewarded with higher productivity. Also, because it is closer to Java than JRuby and Clojure, new users will be able to start using it quickly as a “better object-oriented Java”, while they continue to learn the more advanced features, like functional programming, that will accelerate their productivity over the long term.

Tool Support

Scala support in IDE’s still lags support for Java, but it is improving. IntelliJ, NetBeans, and Eclipse now support Scala with plugins. Maven and ant are widely used as the build tool for Scala applications. Several excellent TDD and BDD libraries are available.

Deployment

Scala applications are packaged and deployed just like Java applications, since Scala files are compiled to class files. A Scala runtime jar is also required.

Clojure

Of the three new JVM languages discussed here, Clojure is the least like Java, due to its Lisp syntax and innovative “programming model”. Yet it is also the most innovative and exciting new JVM language for many people. Clojure interoperates with Java code, but it emphasizes functional programming. Unlike the other languages, Clojure does not support object-oriented programming. Instead, it relies on mechanisms like multi-methods and macros to address design problems for which OOP is often used.

One exciting innovation in Clojure is support for software transactional memory, which uses a database-style transactional approach to concurrent modifications of in-memory, mutable state. STM is somewhat controversial. You can google for arguments about its practicality, etc. However, Clojure’s implementation appears to be successful.

Clojure also has other innovative ways of supporting “principled” modification of mutable data, while encouraging the use of immutable data. These features with STM are the basis of Clojure’s approach to robust concurrency.

Finally, Clojure implements several optimizations in the compiler that are important for functional programming, such as optimizing tail call recursion.

Disclaimer: I know less about Clojure than JRuby and Scala. While I have endeavored to get the facts right, there may be errors in the following analysis. Feedback is welcome.

Interoperability

Despite the Lisp syntax and functional-programming emphasis, Clojure interoperates with Java. Calling java from Clojure uses direct API calls, as for JRuby and Scala. Calling Clojure from Java is a more involved. You have to create Java proxies on the Clojure side to generate the byte code needed on the Java side. The idioms for doing this are straightforward, however.

Object Model

Clojure is not an object-oriented language. However, in order to interoperate with Java code, Clojure supports implementing interfaces and instantiating Java objects. Otherwise, Clojure offers a significant departure for develops well versed in object-oriented programming, but with little functional programming experience.

New Ideas

Clojure brings to the JVM full support for functional programming and popular Lisp concepts like macros, multi-methods, and powerful metaprogramming. It has innovative approaches to safe concurrency, including “principled” mechanisms for supporting mutable state, as discussed previously.

Clojure’s succinct syntax and built-in libraries make processing XML succinct and efficient. DSL creation is also supported using Lisp mechanisms, like macros.

Stability

Clojure is the newest of the three languages profiled here. Hence, it may be the most subject to change. However, given the nature of Lisps, it is more likely that changes will occur in libraries than the language itself. Stability in terms of bugs does not appear to be an issue.

Clojure also has the fewest known production deployments of the three languages. However, industry adoption is expected to happen rapidly.

Performance

Clojure supports type “hints” to assist in optimizing performance. The preliminary discussions I have seen suggest that Clojure offers very good performance.

Adoption

Clojure is more of a departure from Java than is Scala. It will require a motivated team that likes Lisp ;) However, such a team may learn Clojure faster than Scala, since Clojure is a simpler language, e.g., because it doesn’t have its own object model. Also, Lisps are well known for being simple languages, where the real learning comes in understanding how to use it effectively!

However, in my view, as for Scala, the extra learning effort will be rewarded with higher productivity.

Tool Support

As a new language, tool support is limited. Most Clojure developers use Emacs with its excellent Lisp support. Many Java tools can be used with Clojure.

Deployment

Clojure deployment appears to be as straightforward as for the other languages. A Clojure runtime jar is required.

Comparisons

Briefly, let’s review the points and compare the three languages.

Interoperability

All three languages make calling Java code straightforward. Scala interoperates most seamlessly. Scala code is easiest to invoke from Java code, using direct API calls, as long as you know how Scala encodes method names that have “illegal” characters (according to the JVM spec.). Calling JRuby and Clojure code from Java is more involved.

Therefore, if you expect to continue writing Java code that needs to make frequent API calls to the code in the new language, Scala will be a better choice.

Object Model

Scala is closest to Java’s object model. Ruby’s object model is superficially similar to Scala’s, but the dynamic nature of Ruby brings significant differences. Both extend Java’s object model with mixin composition through traits (Scala) or modules (Ruby), that act like interfaces with implementations.

Clojure is quite different, with an emphasis on functional programming and no direct support for object-oriented programming.

New Ideas

JRuby brings the productivity and power of a dynamically-typed language to the JVM, along with the drawbacks. It also brings some functional idioms.

Scala and Clojure bring full support for functional programming. Scala provides a complete Actor model of concurrency (as a library). Clojure brings software transactional memory and other innovations for writing robust concurrent applications. JRuby and Ruby don’t add anything specific for concurrency.

JRuby, like Ruby, is exceptionally good for writing internal DSL’s. Scala is also very good and Clojure benefits from Lisp’s support for DSL creation.

Stability

All the language implementations are of high quality. Scala is the most mature, but JRuby has the widest adoption in production.

Performance

Performance should be comparable for all, but JRuby and Clojure have to deal with some inefficiencies inherent to running dynamic languages on the JVM. Your mileage may vary, so please run realistic profiling experiments on sample implementations that are representative of your needs. Avoid “prematurely optimization” when choosing a new language. Often, team productivity and “time to market” are more important than raw performance.

Adoption

JRuby is the the easiest of the three languages to learn and adopt if you already have some Ruby or Ruby on Rails code in your environment.

Scala has the lowest barrier to adoption because it is the language that most resembles Java “philosophically” (static typing, emphasis on object-oriented programming, etc.). Adopters can start with Scala as a “better Java” and gradually learn the advanced features (mixin composition with traits and functional programming). Scala will appeal the most to teams that prefer statically-typed languages, yet want some of the benefits of dynamically-typed languages, like a succinct syntax.

However, Scala is the most complex of the three languages, while Clojure requires the biggest conceptual leap from Java.

Clojure will appeal to teams willing to explore more radical departures from what they are doing now, with potentially great payoffs!

Deployment

Deployment is easy with all three languages. Scala is most like Java, since you normally compile to class files (there is a limited interpreter mode). JRuby and Clojure code can be interpreted at runtime or compiled.

Summary and Conclusions

All three choices (or comparable substitutions from the list of other languages), will provide a Java team with a more modern language, yet fully leverage the existing investment in Java. Scala is the easiest incremental change. JRuby brings the vibrant Ruby world to the JVM. Clojure offers the most innovative departures from Java.

Writing Java Aspects ... with JRuby and Aquarium! 25

Posted by Dean Wampler Tue, 26 Feb 2008 04:10:00 GMT

Aquarium V0.4.0, my AOP library for Ruby, now supports JRuby. Not only do the regular “pure Ruby” Aquarium specs run reliably under JRuby (V1.1RC2), but you can now write aspects for Java types with Aquarium!

There are some important limitations, though. Cartographers of old would mark dangerous or unknown territory on their maps with hic sunt dracones (“here be dragons”), a reference to the old practice of adorning maps with serpents around the edges.

This is true of Aqurium + Java types in JRuby, too, at least for now.

Aquarium uses Ruby’s metaprogramming API extensively and the JRuby team has done some pretty sophisticated work to integrate Java types with Ruby. Hence, it’s not too surprising there are some gotchas. Hopefully, workarounds will be possible for all of them.

The details are discussed on the JRuby page, the README on the Aquarium site, and of course the “specs” in the distribution’s jruby/spec directory. I’ll summarize them here, after discussing the pros and cons of Aquarium vs. the venerable AspectJ and showing you an example of using Aquarium for Java.

Briefly, Aquarium’s advantages over AspectJ are these:

  1. You can add and remove advice dynamically at runtime. You can’t remove AspectJ advice.
  2. You can advise JDK types easily with Aquarium. AspectJ won’t do this by default, but this is really more of a legacy licensing issue than a real technical limitation.
  3. You can advise individual objects, not just types.

Aquarium’s disadvantages compared to AspectJ include:

  1. Aquarium will be slower than using AspectJ (although this has not been studied in depth yet).
  2. Aquarium’s pointcut language is not as full-featured as AspectJ’s.
  3. There are the bugs and limitations I mentioned above in this initial V0.4.0 release, which I’ll elaborate shortly.

Here is an example of adding tracing calls to a method doIt in all classes that implement the Java interface com.foo.Work.


Aspect.new :before, :calls_to => [:doIt, :do_it], :in_types_and_descendents => Java::com.foo.Work do |jp, obj, *args|
  log "Entering: #{jp.target_type.name}##{jp.method_name}: object = #{object}, args = #{args.inspect}" 
end

There are two important points to notice in this example:

  1. You can choose to refer to the method as do_it (Ruby style) or doIt, but these variants are effectively treated as separate methods; advice on one will not affect invocations of the other. So, if you want to be sure to catch all invocations, use both forms. There is a bug (18326) that happens in certain conditions if you use just the Java naming convention.
  2. If the type is an interface, you must use :types_and_descendents (or one of the supported variants on the word types...). Since interfaces don’t have method implementations, you will match no join points unless you use the _and_descendents clause. (By default, Aquarium warns you when no join points are matched by an aspect.) However, there is a bug (18325) with this approach if Java types are subtyped in Ruby.

Limitations and Bugs

Okay, here’s the “fine print”...

In this (V0.4.0) release, there are some important limitations.

  1. Aquarium advice on a method in a Java type will only be invoked when the method is called directly from Ruby.
  2. To have the advice invoked when the method is called from either Java or Ruby, it is necessary to create a Ruby subclass of the Java type and override the method(s) you want to advise. These overrides can just call super. Note that it will also be necessary for instances of this Ruby type to be used throughout the application, in both the Java and Ruby code. So, you’ll have to instantiate the object in your Ruby code.

Yea, this isn’t so great, but if you’re motivated… ;)

There are also a few outstanding Aquarium bugs (which could actually be JRuby bugs or quirks of the Aquarium-JRuby “interaction”; I’m not yet sure which).

  1. Bug #18325: If you have Ruby subclasses of Java types and you advise a Java method in the hierarchy using :types_and_descendents => MyJavaBaseClassOrInterface and you call unadvise on the aspect, the advice “infrastructure” is not correctly removed from the Ruby types. Workaround: Either don’t “unadvise” such Ruby types or only advise methods in such Ruby types where the method is explicitly overridden in the Ruby class. (The spec and the Rubyforge bug report provide examples.)
  2. Bug #18326: Normally, you can use either Java- or Ruby-style method names (e.g., doSomething vs. do_something), for Java types. However, if you write an aspect using the Java-style for a method name and a Ruby subclass of the Java type where the method is actually defined (i.e., the Ruby class doesn’t override the method), Aquarium acts like the JoinPoint is advised, but the advice is never actually called. Workaround: Use the Ruby-style name in this scenario.

So, there is still some work to do, but it’s promising that you can use an aspect framework in one language with another. A primary goal of Aquarium is to make it easy to write simple aspects. My hope is that people who might find AspectJ daunting will still give Aquarium a try.