Object Mentor Blog: Category Dean's Deprecations

"Programming Scala" Is Now Available

Dean Wampler — Mon, 21 Sep 2009 17:48:00 -0500

Programming Scala is finally available online at Amazon, O’Reilly, Safari, and hopefully at other online and physical bookstores, too. If you have trouble finding it, just memorize the ISBN, 9780596155957, to aid your search. ;)

You can download the complete code examples here. If you want to “try before you buy”, see our labs site.

I pitched the book idea just over one year ago. Alex Payne, my co-author, was also talking to O’Reilly about a book, so we joined forces. It’s been a fast ride.

This book is for you if you are an experienced developer who wants a comprehensive, but fast introduction to Scala. We try to convince you why we like Scala so much, but we also tell you about those dark corners and gotchas that you’ll find in any language. We even preview the forthcoming 2.8 version of Scala. I hope you’ll give Programming Scala a look.

QCon SF 2008: Radical Simplification through Polyglot and Poly-paradigm Programming

Dean Wampler — Tue, 15 Sep 2009 10:35:00 -0500

InfoQ has posted the video of my talk at last year’s QCon San Francisco on Radical Simplification through Polyglot and Poly-paradigm Programming. I make the case that relying on just one programming language or one modularity paradigm (i.e., object-oriented programming, functional programming, etc.) is insufficient for most applications that we’re building today. That includes embedded systems, games, up to complex Internet and enterprise applications.

I’m giving an updated version of this talk at the Strange Loop Conference, October 22-23, in St. Louis. I hope to see you there.

A Milestone for "Programming Scala"

Dean Wampler — Tue, 07 Jul 2009 21:02:00 -0500

My co-author, Alex Payne, and I hit a major milestone today for Programming Scala. After a feverish holiday weekend of writing, we finished all the remaining sections of the book. This morning, we released it to O’Reilly’s crack production team. “If you love something, set it free” or something sappy like that. Of course, the corollary is, “if it doesn’t come back, hunt it down and kill it…”

But I digress. There will be more reviews and final edits. You can still add your comments online using the link above. However, the text is essentially done. It has started the final journey that will turn our words into a book, something of a life-long dream of mine that I waited too long to achieve. Kudo’s to Alex for pursuing this dream early in his career. If it’s a dream you have entertained, know that it’s never too late.

On the other hand, whatever modest qualities the book possesses reflect the combined years of experience that Alex and I have acquired, sometimes painfully. In a way, as I reflect on what I wrote, Programming Scala is a software design book masquerading as a language book. The truly seductive quality of Scala is that it makes elegant design concepts possible, which is why I’ve placed so much faith in the language.

Alex posted his own thoughts on the project, which I hope you’ll take the time to read. I’m grateful for his insights and experience with Scala, the elegance of his prose in the book, and the great work he’s done at Twitter, which caused me to get so addicted to Twitter that I spent too much time tweeting over the last year, which meant I had to work my proverbial butt off this past weekend to get the book done. Thanks a lot!!

Rich Hickey on Testing

Dean Wampler — Fri, 05 Jun 2009 12:40:00 -0500

It was an interesting week at JavaOne, with lots of talks and hallway discussions about new languages on the JVM. One of those languages is Clojure.

Rich Hickey, the creator of Clojure, gave a talk at the Bay Area Clojure User Group Wednesday evening. During the Q&A part, he said that he’s not big on writing tests, although he always runs the tests that other people have written before he commits changes.

Of course, there are many people, including us Object Mentors, who consider TDD to be an essential part of professional software development. Obviously not everyone agrees. James Coplien has been another critic of this view.

We should never accept any dogma. Why is TDD considered important? What does it purport to do? TDD provides two important benefits.

Driving the design.
Building a suite of automated regression tests.

So, if you can satisfy both requirements without TDD, then technically you don’t need it. In Rich’s case, he said he spends a lot of time thinking about what he’s going to do before he does it. In this way, he satisfies the first requirement, driving the design. I had a spirited discussion with some Thoughtworkers afterwards and Ola Bini said what a lot of us think, “I do that thinking by writing tests.” I’ll freely admit that when I am really experimenting with ideas, I might just write code, but once I know how to proceed, I return to the beginning and test drive the “production” code.

Rich also made an off-hand comment that if he screws something up, he’s got thousands of users who will let him know! That ex post facto testing, along with the Rich’s own devotion to doing high-quality work, does a good job of handling regressions.

But Rich mentioned something else that is also very important. In a functional language, where values are immutability and mutable state is handled in specific, principled ways, regressions don’t happen nearly as often. Clojure has one of the most deeply thought out approaches for handling state, which is the genius of Clojure.

I asked Rich how long he worked on Clojure before releasing it to the world. He spent about 2 1/2 years, much of that time working exclusively on Clojure (and eating through his savings). When he finally “announced” it, his “marketing” consisted of one email to some friends in the Common Lisp community. The rest was viral, a testament to the justified excitement Clojure has generated.

For me, I’ll probably always do my design thinking through tests, especially when I’m writing code in imperative languages, like Java and Ruby. I’ll continue to encourage my clients to use TDD, because I find that TDD is the most productive way to achieve high quality. I want the safety net of a good test suite. I’m also writing more and more of my code in a functional style, with minimal side effects and mutable data. You should, too.

Bay-Area Scala Enthusiasts (BASE) Meeting: What's New In Scala 2.8

Dean Wampler — Fri, 05 Jun 2009 02:13:00 -0500

This week is JavaOne in San Francisco. The Bay-Area Scala Enthusiasts (BASE) held their monthly meeting. Martin Odersky, the creator of Scala, was the special guest. He discussed what’s new In Scala 2.8, followed by Q&A. We met at Twitter HQ.

These are my notes, focusing primarily on Martin’s presentation, and filled in afterwards with additional details. Any transcription errors or erroneous extrapolations are my own fault. It’s also late in the day…

Some of the features are not yet in the SVN trunk, so don’t assume my examples actually work! See the scala-lang.org for more details on Scala 2.8 features.

There are a few more months before it is released. A preview is planned for July, followed by the final release in September or October.

New Features

Here are the new features for this release.

Named and Default Arguments

Scala method parameters can be declared to with default values, so callers don’t have to specify a value and the implicit convention doesn’t have to be used. The default “values” aren’t limited to constants. Any valid expression can be used. Here is an example that I made up (not in Martin’s slides) that illustrates both specifying and using one default argument and using named arguments.

    
def joiner(strings: List[String], separator: String = " ") = strings.mkString(separator)

val strs = List("Now", "is", "the", "time", "for", "all", "good", "men", "...")
println(joiner(strs))
println(joiner(strs, "|"))
println(joiner(strings = strs, separator = "-"))

Named and default arguments enable an elegant enhancement to case classes. It’s great that I can declare a succinct value class like this.

    
case class Person(firstName: String, lastName: String, age: Int)

What if I want to make a copy that modifies one or more fields. There’s no elegant way to add such a method in 2.7 without implementing every permutation, that is every possible combination of fields I might want to change. The new copy method will make this easy.

    
case class Person(firstName: String, lastName: String, age: Int)

val youngPerson = Person("Dean", "Wampler", 29)
val oldPerson = youngPerson copy (age = 30)

I’m using the infix notation for method invocation on the last line (i.e., it’s equivalent to ... youngPerson.copy(...)). I can specify any combination of the fields I want to change in the list passed to copy. The generated implementation of copy will use the current values of any other fields as the default values.

The implementation looks something like this.

    
case class Person(firstName: String, lastName: String, age: Int) {
  def copy (fName: String = this.firstName, 
            lName: String = this.lastName, 
            aje: Int = this.age) = new Person(fName, lName, aje)
}

Quite elegant, once you have default and named arguments!!

Defaults for parameters can’t refer to previous parameters in the list, unless the function is curried. (I’m not sure I got this right, nor do I understand the reasons why this is true – if it’s true!)

By the way, Martin reminded us that method parameters are always evaluated left to right at the call site. Do you remember the rules for Java, C++, C#,...?

Nested Annotations

Annotations can now be nested, which is important for using some of the standard annotation definitions in the JDK and JEE. This feature also exploits named and default arguments.

    
@Annotation1(foo = @Annotation2)

Package Objects

People have complained that they want to define top-level definitions for a package, but they have to put those definitions, like types and methods, in an object or class, which doesn’t quite fit and it’s awkward for referencing through package and type qualification. The problem was especially obvious when the team started working on the major reorganization of the collections (discussed below). So, Scala 2.8 will support “package objects”.

    
package object scala {
  type List[+T] = scala.collection.immutable.List
  val List = scala.collection.immutable.List
}

Our friend List is now moved to scala.collection.immutable.List, but we would still like to reference it as if it were in the scala package. The definition defines a package-level type and val the effectively make List accessible in the scala scope. In Scala 2.7 you would have to do something like the following (ignoring Predef for a moment).

    
package scala {
  object toplevel {
    type List[+T] = scala.collection.immutable.List
    val List = scala.collection.immutable.List
  }
}

But then you would have to reference List using scala.toplevel.List.

Now, they got around this problem previously by putting a bunch of stuff like this in Predef and importing it automatically, but that has several disadvantages.

Predef is a big, amorphous collection of stuff.
You can’t define your own Predef with the same convenient usage semantics, i.e., no special import required and no way to reference definitions like package.type. You would have to use the alternative I just showed with toplevel in the middle.

Package objects give you a place for definitions that you want to appear at the package scope without having to define them in a singleton object or class.

Finally, besides types and fields as shown, package objects can also define methods. They can also inherit from traits and classes.

@specialized

Scala generics are fully specified at declaration time using a uniform representation, not when they are used, like C++ templates. This supports the way Java works, where there isn’t a giant link step to resolve all references, etc. However, this has a major performance disadvantage for generic types when they are actually used with AnyVal types that Scala optimizes to primitives.

For example, any closures require the use of FunctionN[T1, T2, ...], e.g.,

    
def m[T](x: T, f: T => T) = f(x)

m(2, (x:Int) => x * 2)

The f closure in the definition of m will require instantiation of Function2[T,T]. However, when use AnyVal classes, as in the last line , this has the effect of causing primitive boxing and unboxing several times, hurting performance when this is completely unnecessary in the special case of primitives being used. This is also bad for arrays and some other data structures.

The new @specialized annotation fixes this problem by causing scala to generate different versions of the user-specified generic type or method for each of the primitive types.

    
def m[@specialized T](x: T, f: T => T) = f(x)

m(2, (x:Int) => x * 2)

There is a real risk of an explosion of code. Consider what would have to be generated to support every type permutation for Function22! For this reason they only do cases with up to two type parameters in the library. You can also choose to annotate only some of the type parameters, as appropriate, and the annotation will support parameters that let you limit the primitive types that will be supported, e.g., only Ints and Longs.

This feature is not yet in the 2.8 trunk, but it will be soon.

Improved Collections

Collections are getting a major revamp. First they want to eliminate gratuitous differences in package structure and implementations. In many cases, the map method and others have to be redefined for each basic collection type, rather than shared between them.

New Collections Design

The new version of the library will support the following.

Uniform structure.
Every operation is implemented only once.
Selection of building blocks in a separate package called scala.collection.generic. These are normally only used by implementers of immutable and mutable collections.

Because of the reorganization, some Scala 2.7 source code won’t be compatible with 2.8 without modifications.

Better Tools

The REPL will have command completion, in addition to other enhancements.
They have greatly improved the IDE and compiler interface. Miles Sabin and Iulian Dragos worked on this with Martin. There is limited and somewhat unstable support in Eclipse now.

New Control Abstractions

Several new control abstractions are being introduced.

Continuations will be supported with a compiler plugin.
Scala has not had the break keyword. It will now exist, but as a library method.
Scala will optimize trampolining tail calls (e.g., foo1 tail calls foo2, which tail calls foo1, and back and forth).

More features

The Swing wrapper library has been enhanced.
The performance has been improved in several ways.
- Structural type dispatch
- Actors
- Vectors, sets, and maps. Their long-term goal is to implement the fastest ones available for the JVM.

These changes are not yet in the trunk.

Beyond 2.8

Longer term, they plan significant improvements in support for parallelism and concurrency, including new concurrency models besides actors, such as:

Transactions (STM)
Data parallelism
stream processing

Clojure is influencing this. Martin praised the competition ;) Fortunately, the original designer of the data structures and algorithms used heavily by Clojure is working on Scala versions. (Name?)

Doug Lea wants to work with the team on concurrency data structures. The lack of closures in Java makes this effort difficult in Java.

There is some exciting work in advanced type system support for guaranteeing actor isolation and effect tracking. For example, this technology wouuld allow actors to exchange references to big objects without copying them while ensuring that they aren’t modified concurrently.

On a final note, Bill Wake described a conversation he had with Joshua Bloch today who admitted that the time has arrived for him to look seriously at Scala. A possible endorsement from Joshua Bloch would be a major step for Scala.

Is the Supremacy of Object-Oriented Programming Over?

Dean Wampler — Mon, 20 Apr 2009 21:45:00 -0500

I never expected to see this. When I started my career, Object-Oriented Programming (OOP) was going mainstream. For many problems, it was and still is a natural way to modularize an application. It grew to (mostly) rule the world. Now it seems that the supremacy of objects may be coming to an end, of sorts.

I say this because of recent trends in our industry and my hands-on experience with many enterprise and Internet applications, mostly at client sites. You might be thinking that I’m referring to the mainstream breakout of Functional Programming (FP), which is happening right now. The killer app for FP is concurrency. We’ve all heard that more and more applications must be concurrent these days (which doesn’t necessarily mean multithreaded). When we remove side effects from functions and disallow mutable variables, our concurrency issues largely go away. The success of the Actor model of concurrency, as used to great effect in Erlang, is one example of a functional-style approach. The rise of map-reduce computations is another example of a functional technique going mainstream. A related phenomenon is the emergence of key-value store databases, like BigTable and CouchDB, is a reaction to the overhead of SQL databases, when the performance cost of the Relational Model isn’t justified. These databases are typically managed with functional techniques, like map-reduce.

But actually, I’m thinking of something else. Hybrid languages like Scala, F#, and OCaml have demonstrated that OOP and FP can complement each other. In a given context, they let you use the idioms that make the most sense for your particular needs. For example, immutable “objects” and functional-style pattern matching is a killer combination.

What’s really got me thinking that objects are losing their supremacy is a very mundane problem. It’s a problem that isn’t new, but like concurrency, it just seems to grow worse and worse.

The problem is that there is never a stable, clear object model in applications any more. What constitutes a BankAccount or Customer or whatever is fluid. It changes with each iteration. It’s different from one subsystem to another even within the same iteration! I see a lot of misfit object models that try to be all things to all people, so they are bloated and the teams that own them can’t be agile. The other extreme is “balkanization”, where each subsystem has its own model. We tend to think the latter case is bad. However, is lean and mean, but non-standard, worse than bloated, yet standardized?

The fact is, for a lot of these applications, it’s just data. The ceremony of object wrappers doesn’t carry its weight. Just put the data in a hash map (or a list if you don’t need the bits “labeled”) and then process the collection with your iterate, map, and reduce functions. This may sound heretical, but how much Java code could you delete today if you replaced it with a stored procedure?

These alternatives won’t work for all situations, of course. Sometimes polymorphism carries its weight. Unfortunately, it’s too tempting to use objects as if more is always better, like cow bell.

So what would replace objects for supremacy? Well, my point is really that there is no one true way. We’ve led ourselves down the wrong path. Or, to be more precise, we followed a single, very good path, but we didn’t know when to take a different path.

Increasingly, the best, most nimble designs I see use objects with a light touch; shallow hierarchies, small objects that try to obey the Single Responsibility Principle, composition rather than inheritance, etc. Coupled with a liberal use of functional idioms (like iterate, map, and reduce), these designs strike the right balance between the protection of data hiding vs. openness for easy processing. By the way, you can build these designs in almost any of our popular languages. Some languages make this easier than others, of course.

Despite the hype, I think Domain-Specific Languages (DSLs) are also very important and worth mentioning in this context. (Language-Oriented Programming – LOP – generalizes these ideas). It’s true that people drink the DSL Kool-Aid and create a mess. However, when used appropriately, DSLs reduce a program to its essential complexity, while hiding and modularizing the accidental complexity of the implementation. When it becomes easy to write a user story in code, we won’t obsess as much over the details of a BankAccount as they change from one story to another. We will embrace more flexible data persistence models, too.

Back to OOP and FP, I see the potential for their combination to lead to a rebirth of the old vision of software components, but that’s a topic for another blog post.

Pat Eyler Interviews Dean Wampler and Alex Payne on "Programming Scala".

Dean Wampler — Tue, 17 Mar 2009 12:48:00 -0500

Pat Eyler posted an interview with Alex Payne and me (Dean Wampler), which we conducted over email. We dish on Scala, Functional Programming, and our forthcoming book Programming Scala.

Tighter Ruby Methods with Functional-style Pattern Matching, Using the Case Gem

Dean Wampler — Mon, 16 Mar 2009 19:59:00 -0500

Ruby doesn’t have overloaded methods, which are methods with the same name, but different signatures when you consider the argument lists and return values. This would be somewhat challenging to support in a dynamic language with very flexible options for method argument handling.

You can “simulate” overloading by parsing the argument list and taking different paths of execution based on the structure you find. This post discusses how pattern matching, a hallmark of functional programming, gives you powerful options.

First, let’s look at a typical example that handles the arguments in an ad hoc fashion. Consider the following Person class. You can pass three arguments to the initializer, the first_name, the last_name, and the age. Or, you can pass a hash using the keys :first_name, :last_name, and :age.


require "rubygems" 
require "spec" 

class Person
  attr_reader :first_name, :last_name, :age
  def initialize *args
    arg = args[0]
    if arg.kind_of? Hash       # 1
      @first_name = arg[:first_name]
      @last_name  = arg[:last_name]
      @age        = arg[:age]
    else
      @first_name = args[0]
      @last_name  = args[1]
      @age        = args[2]
    end
  end
end

describe "Person#initialize" do 
  it "should accept a hash with key-value pairs for the attributes" do
    person = Person.new :first_name => "Dean", :last_name => "Wampler", :age => 39
    person.first_name.should == "Dean" 
    person.last_name.should  == "Wampler" 
    person.age.should        == 39
  end
  it "should accept a first name, last name, and age arguments" do
    person = Person.new "Dean", "Wampler", 39
    person.first_name.should == "Dean" 
    person.last_name.should  == "Wampler" 
    person.age.should        == 39
  end
end

The condition on the # 1 comment line checks to see if the first argument is a Hash. If so, the attribute’s values are extracted from it. Otherwise, it is assumed that three arguments were specified in a particular order. They are passed to #initialize in a three-element array. The two rspec examples exercise these behaviors. For simplicity, we ignore some more general cases, as well as error handling.

Another approach that is more flexible is to use duck typing, instead. For example, we could replace the line with the # 1 comment with this line:


if arg.respond_to? :has_key?

There aren’t many objects that respond to #has_key?, so we’re highly confident that we can use [symbol] to extract the values from the hash.

This implementation is fairly straightforward. You’ve probably written code like this yourself. However, it could get complicated for more involved cases.

Pattern Matching, a Functional Programming Approach

Most programming languages today have switch or case statements of some sort and most have support for regular expression matching. However, in functional programming languages, pattern matching is so important and pervasive that these languages offer very powerful and convenient support for pattern matching.

Fortunately, we can get powerful pattern matching, typical of functional languages, in Ruby using the Case gem that is part of the MenTaLguY’s Omnibus Concurrency library. Omnibus provides support for the hot Actor model of concurrency, which Erlang has made famous. However, it would be a shame to restrict the use of the Case gem to parsing Actor messages. It’s much more general purpose than that.

Let’s rework our example using the Case gem.


require "rubygems" 
require "spec" 
require "case" 

class Person
  attr_reader :first_name, :last_name, :age
  def initialize *args
    case args
    when Case[Hash]       # 1
      arg = args[0]
      @first_name = arg[:first_name]
      @last_name  = arg[:last_name]
      @age        = arg[:age]
    else
      @first_name = args[0]
      @last_name  = args[1]
      @age        = args[2]
    end
  end
end

describe "Person#initialize" do 
  it "should accept a first name, last name, and age arguments" do
    person = Person.new "Dean", "Wampler", 39
    person.first_name.should == "Dean" 
    person.last_name.should  == "Wampler" 
    person.age.should        == 39
  end
  it "should accept a has with :first_name => fn, :last_name => ln, and :age => age" do
    person = Person.new :first_name => "Dean", :last_name => "Wampler", :age => 39
    person.first_name.should == "Dean" 
    person.last_name.should  == "Wampler" 
    person.age.should        == 39
  end
end

We require the case gem, which puts the #=== method on steroids. In the when statement in #initialize, the expression when Case[Hash] matches on a one-element array where the element is a Hash. We extract the key-value pairs as before. The else clause assumes we have an array for the arguments.

So far, this is isn’t very impressive, but all we did was to reproduce the original behavior. Let’s extend the example to really exploit some of the neat features of the Case gem’s pattern matching. First, let’s narrow the allowed array values.


require "rubygems" 
require "spec" 
require "case" 

class Person
  attr_reader :first_name, :last_name, :age
  def initialize *args
    case args
    when Case[Hash]       # 1
      arg = args[0]
      @first_name = arg[:first_name]
      @last_name  = arg[:last_name]
      @age        = arg[:age]
    when Case[String, String, Integer]
      @first_name = args[0]
      @last_name  = args[1]
      @age        = args[2]
    else
      raise "Invalid arguments: #{args}" 
    end
  end
end

describe "Person#initialize" do 
  it "should accept a first name, last name, and age arguments" do
    person = Person.new "Dean", "Wampler", 39
    person.first_name.should == "Dean" 
    person.last_name.should  == "Wampler" 
    person.age.should        == 39
  end
  it "should accept a has with :first_name => fn, :last_name => ln, and :age => age" do
    person = Person.new :first_name => "Dean", :last_name => "Wampler", :age => 39
    person.first_name.should == "Dean" 
    person.last_name.should  == "Wampler" 
    person.age.should        == 39
  end
  it "should not accept an array unless it is a [String, String, Integer]" do
    lambda { person = Person.new "Dean", "Wampler", "39" }.should raise_error(Exception)
  end
end

The new expression when Case[String, String, Integer] only matches a three-element array where the first two arguments are strings and the third argument is an integer, which are the types we want. If you use an array with a different number of arguments or the arguments have different types, this when clause won’t match. Instead, you’ll get the default else clause, which raises an exception. We added another rspec example to test this condition, where the user’s age was specified as a string instead of as an integer. Of course, you could decide to attempt a conversion of this argument, to make your code more “forgiving” of user mistakes.

Similarly, what happens if the method supports default values some of the parameters. As written, we can’t support that option, but let’s look at a slight variation of Person#initialize, where a hash of values is not supported, to see what would happen.


require "rubygems" 
require "spec" 
require "case" 

class Person
  attr_reader :first_name, :last_name, :age
  def initialize first_name = "Bob", last_name = "Martin", age = 29
    case [first_name, last_name, age]
    when Case[String, String, Integer]
      @first_name = first_name
      @last_name  = last_name
      @age        = age
    else
      raise "Invalid arguments: #{first_name}, #{last_name}, #{age}" 
    end
  end
end

def check person, expected_fn, expected_ln, expected_age
  person.first_name.should == expected_fn
  person.last_name.should  == expected_ln
  person.age.should        == expected_age
end

describe "Person#initialize" do 
  it "should require a first name (string), last name (string), and age (integer) arguments" do
    person = Person.new "Dean", "Wampler", 39
    check person, "Dean", "Wampler", 39
  end
  it "should accept the defaults for all parameters" do
    person = Person.new
    check person, "Bob", "Martin", 29
  end
  it "should accept the defaults for the last name and age parameters" do
    person = Person.new "Dean" 
    check person, "Dean", "Martin", 29
  end
  it "should accept the defaults for the age parameter" do
    person = Person.new "Dean", "Wampler" 
    check person, "Dean", "Wampler", 29
  end
  it "should not accept the first name as a symbol" do
    lambda { person = Person.new :Dean, "Wampler", "39" }.should raise_error(Exception)
  end
  it "should not accept the last name as a symbol" do
  end
  it "should not accept the age as a string" do
    lambda { person = Person.new "Dean", "Wampler", "39" }.should raise_error(Exception)
  end
end

We match on all three arguments as an array, asserting they are of the correct type. As you might expect, #initialize always gets three parameters passed to it, including when default values are used.

Let’s return to our original example, where the object can be constructed with a hash or a list of arguments. There are two more things (at least …) that we can do. First, we’re not yet validating the types of the values in the hash. Second, we can use the Case gem to impose constraints on the values, such as requiring non-empty name strings and a positive age.


require "rubygems" 
require "spec" 
require "case" 

class Person
  attr_reader :first_name, :last_name, :age
  def initialize *args
    case args
    when Case[Hash]
      arg = args[0]
      @first_name = arg[:first_name]
      @last_name  = arg[:last_name]
      @age        = arg[:age]
    when Case[String, String, Integer]
      @first_name = args[0]
      @last_name  = args[1]
      @age        = args[2]
    else
      raise "Invalid arguments: #{args}" 
    end
    validate_name @first_name, "first_name" 
    validate_name @last_name, "last_name" 
    validate_age
  end

  protected

  def validate_name name, field_name
    case name
    when Case::All[String, Case.guard {|s| s.length > 0 }]
    else
      raise "Invalid #{field_name}: #{first_name}" 
    end
  end

  def validate_age
    case @age
    when Case::All[Integer, Case.guard {|n| n > 0 }]
    else
      raise "Invalid age: #{@age}" 
    end
  end
end

describe "Person#initialize" do 
  it "should accept a first name, last name, and age arguments" do
    person = Person.new "Dean", "Wampler", 39
    person.first_name.should == "Dean" 
    person.last_name.should  == "Wampler" 
    person.age.should        == 39
  end
  it "should accept a has with :first_name => fn, :last_name => ln, and :age => age" do
    person = Person.new :first_name => "Dean", :last_name => "Wampler", :age => 39
    person.first_name.should == "Dean" 
    person.last_name.should  == "Wampler" 
    person.age.should        == 39
  end
  it "should not accept an array unless it is a [String, String, Integer]" do
    lambda { person = Person.new "Dean", "Wampler", "39" }.should raise_error(Exception)
  end
  it "should not accept a first name that is a zero-length string" do
    lambda { person = Person.new "", "Wampler", 39 }.should raise_error(Exception)
  end    
  it "should not accept a first name that is not a string" do
    lambda { person = Person.new :Dean, "Wampler", 39 }.should raise_error(Exception)
  end    
  it "should not accept a last name that is a zero-length string" do
    lambda { person = Person.new "Dean", "", 39 }.should raise_error(Exception)
  end    
  it "should not accept a last name that is not a string" do
    lambda { person = Person.new :Dean, :Wampler, 39 }.should raise_error(Exception)
  end    
  it "should not accept an age that is less than or equal to zero" do
    lambda { person = Person.new "Dean", "Wampler", -1 }.should raise_error(Exception)
    lambda { person = Person.new "Dean", "Wampler", 0 }.should raise_error(Exception)
  end    
  it "should not accept an age that is not an integer" do
    lambda { person = Person.new :Dean, :Wampler, "39" }.should raise_error(Exception)
  end    
end

We have added validate_name and validate_age methods that are invoked at the end of #initialize. In validate_name, the one when clause requires “all” the conditions to be true, that the name is a string and that it has a non-zero length. Similarly, validate_age has a when clause that requires age to be a positive integer.

Final Thoughts

So, how valuable is this? The code is certainly longer, but it specifies and enforces expected behavior more precisely. The rspec examples verify the enforcement. It smells a little of static typing, which is good or bad, depending on your point of view. ;)

Personally, I think the conditional checks are a good way to add robustness in small ways to libraries that will grow and evolve for a long time. The checks document the required behavior for code readers, like new team members, but of course, they should really get that information from the tests. ;) (However, it would be nice to extract the information into the rdocs.)

For small, short-lived projects, I might not worry about the conditional checks as much (but how many times have those “short-lived projects” refused to die?).

You can read more about Omnibus and Case in this InfoQ interview with MenTaLguY. I didn’t discuss using the Actor model of concurrency, for which these gems were designed. For an example of Actors using Omnibus, see my Better Ruby through Functional Programming presentation or the Confreak’s video of an earlier version of the presentation I gave at last year’s RubyConf.

1st Ever Chicago Area Scala Enthusiasts (CASE) Meeting Tonight

Dean Wampler — Thu, 19 Feb 2009 18:17:00 -0600

Tonight is our first meeting, at the ThoughtWorks offices in the Aon building downtown. If you’re going and you haven’t RSVP’ed, either send a tweet to @chicagoscala or reply here ASAP!

Hope to see you there. Our meetings will be the 3rd Thursday of each month.

Organizing a Chicago Area Scala Enthusiasts (CASE) Group

Dean Wampler — Sat, 17 Jan 2009 17:02:00 -0600

I’m organizing a group in Chicago for people interested in Scala, called the Chicago Area Scala Enthusiasts (CASE). If you’re interested, join the google group for more information.

Adopting New JVM Languages in the Enterprise (Update)

Dean Wampler — Thu, 15 Jan 2009 01:40:00 -0600

(Updated to add Groovy, which I should have mentioned the first time. Also mentioned Django under Python.)

This is an exciting time to be a Java programmer. The pace of innovation for the Java language is slowing down, in part due to concerns that the language is growing too big and in part due to economic difficulties at Sun, which means there are fewer developers assigned to Java. However, the real crown jewel of the Java ecosystem, the JVM, has become an attractive platform for new languages. These languages give us exciting new opportunities for growth, while preserving our prior investment in code and deployment infrastructure.

This post emphasizes practical issues of evaluating and picking new JVM languages for an established Java-based enterprise.

The Interwebs are full of technical comparisons between Java and the different languages, e.g., why language X fixes Java’s perceived issue Y. I won’t rehash those arguments here, but I will describe some language features, as needed.

A similar “polyglot” trend is happening on the .NET platform.

The New JVM Languages

I’ll limit my discussion to these representative (and best known) alternative languages for the JVM.

JRuby – Ruby running on the JVM.
Scala – A hybrid object-oriented and functional language that runs on .NET as well as the JVM. (Disclaimer: I’m co-writing a book on Scala for O’Reilly.)
Clojure – A Lisp dialect.

I picked these languages because they seem to be the most likely candidates for most enterprises considering a new JVM language, although some of the languages listed below could make that claim.

There are other deserving languages besides these three, but I don’t have the time to do them justice. Hopefully, you can generalize the subsequent discussion for these other languages.

Groovy – A dynamically-typed language designed specifically for interoperability with Java. It will appeal to teams that want a dynamically-typed language that is closer to Java than Ruby. With Grails, you have a combination that’s comparable to Ruby on Rails.
Jython – The first non-Java language ported to the JVM, started by Jim Hugunin in 1997. Most of my remarks about JRuby are applicable to Jython. Django is the Python analog of Rails. If your Java shop already has a lot of Python, consider Jython.
Fan – A hybrid object-oriented and functional language that runs on .NET, too. It has a lot of similarities to Scala, like a scripting-language feel.
Ioke – (pronounced “eye-oh-key”) An innovative language developed by Ola Bini and inspired by Io and Lisp. This is the newest language discussed here. Hence, it has a small following, but a lot of potential. The Io/Lisp-flavored syntax will be more challenging to average Java developers than Scala, JRuby, Jython, Fan, and JavaScript.
JavaScript, e.g., Rhino – Much maligned and misunderstood (e.g., due to buggy and inconsistent browser implementations), JavaScript continues to gain converts as an alternative scripting language for Java applications. It is the default scripting language supported by the JDK 6 scripting interface.
Fortress – A language designed as a replacement for high-performance FORTRAN for industrial and academic “number crunching”. This one will interest scientists and engineers…

Note: Like a lot of people, I use the term scripting language to refer to languages with a lightweight syntax, usually dynamically typed. The name reflects their convenience for “scripting”, but that quality is sometimes seen as pejorative; they aren’t seen as “serious” languages. I reject this view.

To learn more about what people are doing on the JVM today (with some guest .NET presentations), a good place to start is the recent JVM Language Summit.

Criteria For Evaluating New JVM Languages

I’ll frame the discussion around a few criteria you should consider when evaluating language choices. I’ll then discuss how each of the languages address those criteria. Since we’re restricting ourselves to JVM languages, I assume that each language compiles to valid byte code, so code in the new language and code written in Java can call each other, at least at some level. The “some level” part will be one criterion. Substitute X for the language you are considering.

Interoperability: How easily can X code invoke Java code and vice versa? Specifically:
1. Create objects (i.e., call new Foo(...)).
2. Call methods on an object.
3. Call static methods on a class.
4. Extend a class.
5. Implement an interface.
Object Model: How different is the object model of X compared to Java’s object model? (This is somewhat tied to the previous point.)
New “Ideas”: Does X support newer programming trends:
1. Functional Programming.
2. Metaprogramming.
3. Easier approaches to writing robust concurrent applications.
4. Easier support for processing XML, SQL queries, etc.
5. Support internal DSL creation.
6. Easier presentation-tier development of web and thick-client UI’s.
Stability: How stable is the language, in terms of:
1. Lack of Bugs.
2. Stability of the language’s syntax, semantics, and library API’s. (All the languages can call Java API’s.)
Performance: How does code written in X perform?
Adoption: Is X easy to learn and use?
Tool Support: What about editors, IDE’s, code coverage, etc.
Deployment: How are apps and libraries written in X deployed?
1. Do I have to modify my existing infrastructure, management, etc.?

The Interoperability point affects ease of adoption and use with a legacy Java code base. The Object Model and Adoption points address the barrier to adoption from the learning point of view. The New “Ideas” point asks what each language brings to development that is not available in Java (or poorly supported) and is seen as valuable to the developer. Finally, Stability, Performance, and Deployment address very practical issues that a candidate production language must address.

Comparing the Languages

JRuby

JRuby is the most popular alternative JVM langauge, driven largely by interest in Ruby and Ruby on Rails.

Interoperability

Ruby’s object model is a little different than Java’s, but JRuby provides straightforward coding idioms that make it easy to call Java from JRuby. Calling JRuby from Java requires the JSR 223 scripting interface or a similar approach, unless JRuby is used to compile the Ruby code to byte code first. In that case, shortcuts are possible, which are well documented.

Object Model

Ruby’s object model is a little different than Java’s. Ruby support mixin-style modules, which behave like interfaces with implementations. So, the Ruby object model needs to be learned, but it is straightforward or the Java developer.

New Ideas

JRuby brings closures to the JVM, a much desired feature that probably won’t be added in the forthcoming Java 7. Using closures, Ruby supports a number of functional-style iterative operations, like mapping, filtering, and reducing/folding. However, Ruby does not fully support functional programming.

Ruby uses dynamic-typing instead of static-typing, which it exploits to provide extensive and powerful metaprogramming facilities.

Ruby doesn’t offer any specific enhancements over Java for safe, robust concurrent programming.

Ruby API’s make XML processing and database access relatively easy. Ruby on Rails is legendary for improving the productivity of web developers and similar benefits are available for thick-client developers using other libraries.

Ruby is also one of the best languages for defining “internal” DSL’s, which are used to great affect in Rails (e.g., ActiveRecord).

Stability

JRuby and Ruby are very stable and are widely used in production. JRuby is believed to be the best performing Ruby platform.

The Ruby syntax and API are undergoing some significant changes in the current 1.9.X release, but migration is not a major challenge.

Performance

JRuby is believed to be the best performing Ruby platform. While it is a topic of hot debate, Ruby and most dynamically-typed languages have higher runtime overhead compared to statically-typed languages. Also, the JVM has some known performance issues for dynamically-typed languages, some of which will be fixed in JDK 7.

As always, enterprises should profile code written in their languages of choice to pick the best one for each particular task.

Adoption

Ruby is very easy to learn, although effective use of advanced techniques like metaprogramming require some time to master. JRuby-specific idioms are also easy to master and are well documented.

Tool Support

Ruby is experiencing tremendous growth in tool support. IDE support still lags support for Java, but IntelliJ, NetBeans, and Eclipse are working on Ruby support. JRuby users can exploit many Java tools.

Code analysis tools and testing tools (TDD and BDD styles) are now better than Java’s.

Deployment

JRuby applications, even Ruby on Rails applications, can be deployed as jars or wars, requiring no modifications to an existing java-based infrastructure. Teams use this approach to minimize the “friction” of adopting Ruby, while also getting the performance benefits of the JVM.

Because JRuby code is byte code at runtime, it can be managed with JMX, etc.

Scala

Scala is a statically-typed language that supports an improved object model (with a full mixin mechanism called traits; similar to Ruby modules) and full support for functional programming, following a design goal of the inventor of Scala, Martin Odersky, that these two paradigms can be integrated, despite some surface incompatibilities. Odersky was involved in the design of Java generics (through earlier research languages) and he wrote the original version of the current javac. The name is a contraction of “scalable language”, but the first “a” is pronounced like “ah”, not long as in the word “hay”.

The syntax looks like a cross between Ruby (method definitions start with the def keyword) and Java (e.g., curly braces). Type inferencing and other syntactic conventions significantly reduce the “cluuter”, such as the number of explicit type declarations (“annotations”) compared to Java. Scala syntax is very succinct, sometimes even more so than Ruby! For more on Scala, see also my previous blog postings, part 1, part 2, part 3, and this related post on traits vs. aspects.

Interoperability

Scala’s has the most seamless interoperability with Java of any of the languages discussed here. This is due in part to Scala’s static typing and “closed” classes (as opposed to Ruby’s “open” classes). It is trivial to import and use Java classes, implement interfaces, etc.

Direct API calls from Java to Scala are also supported. The developer needs to know how the names of Scala methods are encoding in byte code. For example, Scala methods can have “operator” names, like ”+”. In the byte code, that name will be ”$plus”.

Object Model

Scala’s object model extends Java’s model with traits, which support flexble mixin composition. Traits behave like interfaces with implementations. The Scala object model provides other sophisticated features for building “scalable applications”.

New Ideas

Scala brings full support for functional programming to the JVM, including first-class function and closures. Other aspects of functional programming, like immutable variables and side-effect free functions, are encouraged by the language, but not mandated, as Scala is not a pure functional language. (Functional programming is very effective strategy for writing tread-safe programs, etc.) Scala’s Actor library is a port of Erlang’s Actor library, a message-based concurrency approach.

In my view, the Actor model is the best general-purpose approach to concurrency. There are times when multi-threaded code is needed for performance, but not for most concurrent applications. (Note: there are Actor libraries for Java, e.g., Kilim.)

Scala has very good support for building internal DSL’s, although it is not quite as good as Ruby’s features for this purpose. It has a combinator parser library that makes external DSL creation comparatively easy. Scala also offers some innovative API’s for XML processing and Swing development.

Stability

Scala is over 5 years old and it is very stable. The API and syntax continue to evolve, but no major, disruptive changes are expected. In fact, the structure of the language is such that almost all changes occur in libraries, not the language grammar.

There are some well-known production deployments, such as back-end services at twitter.

Performance

Scala provides comparable performance to Java, since it is very close “structurally” to Java code at the byte-code level, given the static typing and “closed” classes. Hence, Scala can exploit JVM optimizations that aren’t available to dynamically-typed languages.

However, Scala will also benefit from planned improvements to support dynamically-typed languages, such as tail-call optimizations (which Scala current does in the compiler.) Hence, Scala probably has marginally better performance than JRuby, in general. If true, Scala may be more appealing than JRuby as a general-purpose, systems language, where performance is critical.

Adoption

Scala is harder to learn and master than JRuby, because it is a more comprehensive language. It not only supports a sophisticated object model, but it also supports functional programming, type inferencing, etc. In my view, the extra effort will be rewarded with higher productivity. Also, because it is closer to Java than JRuby and Clojure, new users will be able to start using it quickly as a “better object-oriented Java”, while they continue to learn the more advanced features, like functional programming, that will accelerate their productivity over the long term.

Tool Support

Scala support in IDE’s still lags support for Java, but it is improving. IntelliJ, NetBeans, and Eclipse now support Scala with plugins. Maven and ant are widely used as the build tool for Scala applications. Several excellent TDD and BDD libraries are available.

Deployment

Scala applications are packaged and deployed just like Java applications, since Scala files are compiled to class files. A Scala runtime jar is also required.

Clojure

Of the three new JVM languages discussed here, Clojure is the least like Java, due to its Lisp syntax and innovative “programming model”. Yet it is also the most innovative and exciting new JVM language for many people. Clojure interoperates with Java code, but it emphasizes functional programming. Unlike the other languages, Clojure does not support object-oriented programming. Instead, it relies on mechanisms like multi-methods and macros to address design problems for which OOP is often used.

One exciting innovation in Clojure is support for software transactional memory, which uses a database-style transactional approach to concurrent modifications of in-memory, mutable state. STM is somewhat controversial. You can google for arguments about its practicality, etc. However, Clojure’s implementation appears to be successful.

Clojure also has other innovative ways of supporting “principled” modification of mutable data, while encouraging the use of immutable data. These features with STM are the basis of Clojure’s approach to robust concurrency.

Finally, Clojure implements several optimizations in the compiler that are important for functional programming, such as optimizing tail call recursion.

Disclaimer: I know less about Clojure than JRuby and Scala. While I have endeavored to get the facts right, there may be errors in the following analysis. Feedback is welcome.

Interoperability

Despite the Lisp syntax and functional-programming emphasis, Clojure interoperates with Java. Calling java from Clojure uses direct API calls, as for JRuby and Scala. Calling Clojure from Java is a more involved. You have to create Java proxies on the Clojure side to generate the byte code needed on the Java side. The idioms for doing this are straightforward, however.

Object Model

Clojure is not an object-oriented language. However, in order to interoperate with Java code, Clojure supports implementing interfaces and instantiating Java objects. Otherwise, Clojure offers a significant departure for develops well versed in object-oriented programming, but with little functional programming experience.

New Ideas

Clojure brings to the JVM full support for functional programming and popular Lisp concepts like macros, multi-methods, and powerful metaprogramming. It has innovative approaches to safe concurrency, including “principled” mechanisms for supporting mutable state, as discussed previously.

Clojure’s succinct syntax and built-in libraries make processing XML succinct and efficient. DSL creation is also supported using Lisp mechanisms, like macros.

Stability

Clojure is the newest of the three languages profiled here. Hence, it may be the most subject to change. However, given the nature of Lisps, it is more likely that changes will occur in libraries than the language itself. Stability in terms of bugs does not appear to be an issue.

Clojure also has the fewest known production deployments of the three languages. However, industry adoption is expected to happen rapidly.

Performance

Clojure supports type “hints” to assist in optimizing performance. The preliminary discussions I have seen suggest that Clojure offers very good performance.

Adoption

Clojure is more of a departure from Java than is Scala. It will require a motivated team that likes Lisp ;) However, such a team may learn Clojure faster than Scala, since Clojure is a simpler language, e.g., because it doesn’t have its own object model. Also, Lisps are well known for being simple languages, where the real learning comes in understanding how to use it effectively!

However, in my view, as for Scala, the extra learning effort will be rewarded with higher productivity.

Tool Support

As a new language, tool support is limited. Most Clojure developers use Emacs with its excellent Lisp support. Many Java tools can be used with Clojure.

Deployment

Clojure deployment appears to be as straightforward as for the other languages. A Clojure runtime jar is required.

Comparisons

Briefly, let’s review the points and compare the three languages.

Interoperability

All three languages make calling Java code straightforward. Scala interoperates most seamlessly. Scala code is easiest to invoke from Java code, using direct API calls, as long as you know how Scala encodes method names that have “illegal” characters (according to the JVM spec.). Calling JRuby and Clojure code from Java is more involved.

Therefore, if you expect to continue writing Java code that needs to make frequent API calls to the code in the new language, Scala will be a better choice.

Object Model

Scala is closest to Java’s object model. Ruby’s object model is superficially similar to Scala’s, but the dynamic nature of Ruby brings significant differences. Both extend Java’s object model with mixin composition through traits (Scala) or modules (Ruby), that act like interfaces with implementations.

Clojure is quite different, with an emphasis on functional programming and no direct support for object-oriented programming.

New Ideas

JRuby brings the productivity and power of a dynamically-typed language to the JVM, along with the drawbacks. It also brings some functional idioms.

Scala and Clojure bring full support for functional programming. Scala provides a complete Actor model of concurrency (as a library). Clojure brings software transactional memory and other innovations for writing robust concurrent applications. JRuby and Ruby don’t add anything specific for concurrency.

JRuby, like Ruby, is exceptionally good for writing internal DSL’s. Scala is also very good and Clojure benefits from Lisp’s support for DSL creation.

Stability

All the language implementations are of high quality. Scala is the most mature, but JRuby has the widest adoption in production.

Performance

Performance should be comparable for all, but JRuby and Clojure have to deal with some inefficiencies inherent to running dynamic languages on the JVM. Your mileage may vary, so please run realistic profiling experiments on sample implementations that are representative of your needs. Avoid “prematurely optimization” when choosing a new language. Often, team productivity and “time to market” are more important than raw performance.

Adoption

JRuby is the the easiest of the three languages to learn and adopt if you already have some Ruby or Ruby on Rails code in your environment.

Scala has the lowest barrier to adoption because it is the language that most resembles Java “philosophically” (static typing, emphasis on object-oriented programming, etc.). Adopters can start with Scala as a “better Java” and gradually learn the advanced features (mixin composition with traits and functional programming). Scala will appeal the most to teams that prefer statically-typed languages, yet want some of the benefits of dynamically-typed languages, like a succinct syntax.

However, Scala is the most complex of the three languages, while Clojure requires the biggest conceptual leap from Java.

Clojure will appeal to teams willing to explore more radical departures from what they are doing now, with potentially great payoffs!

Deployment

Deployment is easy with all three languages. Scala is most like Java, since you normally compile to class files (there is a limited interpreter mode). JRuby and Clojure code can be interpreted at runtime or compiled.

Summary and Conclusions

All three choices (or comparable substitutions from the list of other languages), will provide a Java team with a more modern language, yet fully leverage the existing investment in Java. Scala is the easiest incremental change. JRuby brings the vibrant Ruby world to the JVM. Clojure offers the most innovative departures from Java.

Video of my RubyConf talk, "Better Ruby through Functional Programming"

Dean Wampler — Thu, 27 Nov 2008 16:09:00 -0600

Confreaks has started posting the videos from RubyConf. Here’s mine on Better Ruby through Functional Programming.

Please ignore the occasional Ruby (and Scala) bugs…

Upcoming Speaking Engagements

Dean Wampler — Mon, 03 Nov 2008 18:13:00 -0600

I’m speaking this Friday at RubyConf on Better Ruby Through Functional Programming. I’ll introduce long-overlooked ideas from FP, why they are important for Ruby programmers, and how to use them in Ruby.

In two weeks, I’m speaking on Wednesday, 11/19 at QCon San Francisco on Radical Simplification Through Polyglot and Poly-paradigm Programming. The idea of this talk is that combining the right languages and modularity paradigms (i.e., objects, functions, aspects) can simplify your code base and reduce the amount of code you have to write and manage, providing numerous benefits.

Back in Chicago, I’m speaking at the Polyglot Programmer’s meeting on The Seductions of Scala, 11/13. It’s an intro to the Scala language, which could become the language of choice for the JVM. I’m repeating this talk at the Chicago Java User’s Group on 12/16. I’m co-writing a book on Scala with Alex Payne. O’Reilly will be the publisher.

Incidently, Bob Martin is also speaking in Chicago on 11/13 at the APLN Chicago meeting on Software Professionalism.

A Scala-style "with" Construct for Ruby

Dean Wampler — Mon, 29 Sep 2008 22:41:00 -0500

Scala has a “mixin” construct called traits, which are roughly analogous to Ruby modules. They allow you to create reusable, modular bits of state and behavior and use them to compose classes and other traits or modules.

The syntax for using Scala traits is quite elegant. It’s straightforward to implement the same syntax in Ruby and doing so has a few useful advantages.

For example, here is a Scala example that uses a trait to trace calls to a Worker.work method.

    
// run with "scala example.scala" 

class Worker {
    def work() = "work" 
}

trait WorkerTracer extends Worker {
    override def work() = "Before, " + super.work() + ", After" 
}

val worker = new Worker with WorkerTracer

println(worker.work())        // => Before, work, After

Note that WorkerTracer extends Worker so it can override the work method. Since Scala is statically typed, you can’t just define an override method and call super unless the compiler knows there really is a “super” method!

Here’s a Ruby equivalent.

    
# run with "ruby example.rb" 

module WorkerTracer
    def work; "Before, #{super}, After"; end
end

class Worker 
    def work; "work"; end
end

class TracedWorker < Worker 
  include WorkerTracer
end

worker = TracedWorker.new

puts worker.work          # => Before, work, After

Note that we have to create a subclass, which isn’t required for the Scala case (but can be done when desired).

If you know that you will always want to trace calls to work in the Ruby case, you might be tempted to dispense with the subclass and just add include WorkerTracer in Worker. Unfortunately, this won’t work. Due to the way that Ruby resolves methods, the version of work in the module will not be found before the version defined in Worker itself. Hence the subclass seems to be the only option.

However, we can work around this using metaprogramming. We can use WorkerTracer#append_features(...). What goes in the argument list? If we pass Worker, then all instances of Worker will be effected, but actually we’ll still have the problem with the method resolution rules.

If we just want to affect one object and work around the method resolution roles, then we need to pass the singleton class (or eigenclass or metaclass ...) for the object, which you can get with the following expression.

    
metaclass = class << worker; self; end

So, to encapsulate all this and to get back to the original goal of implementing with-style semantics, here is an implementation that adds a with method to Object, wrapped in an rspec example.

    
# run with "spec ruby_with_spec.rb" 

require 'rubygems'
require 'spec'

# Warning, monkeypatching Object, especially with a name
# that might be commonly used is fraught with peril!!

class Object
  def with *modules
    metaclass = class << self; self; end
    modules.flatten.each do |m|
      m.send :append_features, metaclass
    end
    self
  end
end

module WorkerTracer
    def work; "Before, #{super}, After"; end
end

module WorkerTracer1
    def work; "Before1, #{super}, After1"; end
end

class Worker 
    def work; "work"; end
end

describe "Object#with" do
  it "should make no changes to an object if no modules are specified" do
    worker = Worker.new.with
    worker.work.should == "work" 
  end

  it "should override any methods with a module's methods of the same name" do
    worker = Worker.new.with WorkerTracer
    worker.work.should == "Before, work, After" 
  end

  it "should stack overrides for multiple modules" do
    worker = Worker.new.with(WorkerTracer).with(WorkerTracer1)
    worker.work.should == "Before1, Before, work, After, After1" 
  end

  it "should stack overrides for a list of modules" do
    worker = Worker.new.with WorkerTracer, WorkerTracer1
    worker.work.should == "Before1, Before, work, After, After1" 
  end

  it "should stack overrides for an array of modules" do
    worker = Worker.new.with [WorkerTracer, WorkerTracer1]
    worker.work.should == "Before1, Before, work, After, After1" 
  end
end

You should carefully consider the warning about monkeypatching Object! Also, note that Module.append_features is actually private, so I had to use m.send :append_features, ... instead.

The syntax is reasonably intuitive and it eliminates the need for an explicit subclass. You can pass a single module, or a list or array of them. Because with returns the object, you can also chain with calls.

A final note; many developers steer clear of metaprogramming and reflection features in their languages, out of fear. While prudence is definitely wise, the power of these tools can dramatically accelerate your productivity. Metaprogramming is just programming. Every developer should master it.

Traits vs. Aspects in Scala

Dean Wampler — Sat, 27 Sep 2008 22:33:00 -0500

Scala traits provide a mixin composition mechanism that has been missing in Java. Roughly speaking, you can think of traits as analogous to Java interfaces, but with implementations.

Aspects, e.g., those written in AspectJ, are another mechanism for mixin composition in Java. How do aspects and traits compare?

Let’s look at an example trait first, then re-implement the same behavior using an AspectJ aspect, and finally compare the two approaches.

Observing with Traits

In a previous post on Scala, I gave an example of the Observer Pattern implemented using a trait. Chris Shorrock and James Iry provided improved versions in the comments. I’ll use James’ example here.

To keep things as simple as possible, let’s observe a simple Counter, which increments an internal count variable by the number input to an add method.

    
package example

class Counter {
    var count = 0
    def add(i: Int) = count += i
}

The count field is actually public, but I will only write to it through add.

Here is James’ Subject trait that implements the Observer Pattern.

    
package example

trait Subject {
  type Observer = { def receiveUpdate(subject:Any) }

  private var observers = List[Observer]()
  def addObserver(observer:Observer) = observers ::= observer
  def notifyObservers = observers foreach (_.receiveUpdate(this))
}

Effectively, this says that we can use any object as an Observer as long as it matches the structural type { def receiveUpdate(subject:Any) }. Think of structural types as anonymous interfaces. Here, a valid observer is one that has a receiveUpdate method taking an argument of Any type.

The rest of the trait manages a list of observers and defines a notifyObservers method. The expression observers ::= observer uses the List :: (“cons”) operator to prepend an item to the list. (Note, I am using the default immutable List, so a new copy is created everytime.)

The notifyObservers method iterates through the observers, calling receiveUpdate on each one. The _ that gets replaced with each observer during the iteration.

Finally, here is a specs file that exercises the code.

    
package example

import org.specs._

object CounterObserverSpec extends Specification {
    "A Counter Observer" should {
        "observe counter increments" in {
            class CounterObserver {
                var updates = 0
                def receiveUpdate(subject:Any) = updates += 1
            }
            class WatchedCounter extends Counter with Subject {
                override def add(i: Int) = { 
                    super.add(i)
                    notifyObservers
                }
            }
            var watchedCounter = new WatchedCounter
            var counterObserver = new CounterObserver
            watchedCounter.addObserver(counterObserver)
            for (i <- 1 to 3) watchedCounter.add(i)
            counterObserver.updates must_== 3
            watchedCounter.count must_== 6
    }
  }
}

The specs library is a BDD tool inspired by rspec in Rubyland.

I won’t discuss it all the specs-specific details here, but hopefully you’ll get the general idea of what it’s doing.

Inside the "observe counter increments" in {...}, I start by declaring two classes, CounterObserver and WatchedCounter. CounterObserver satisfies our required structural type, i.e., it provides a receiveUpdate method.

WatchedCounter subclasses Counter and mixes in the Subject trait. It overrides the add method, where it calls Counter’s add first, then notifies the observers. No parentheses are used in the invocation of notifyObservers because the method was not defined to take any!

Next, I create an instance of each class, add the observer to the WatchedCounter, and make 3 calls to watchedCounter.add.

Finally, I use the “actual must_== expected” idiom to test the results. The observer should have seen 3 updates, while the counter should have a total of 6.

The following simple bash shell script will build and run the code.

    
SCALA_HOME=...
SCALA_SPECS_HOME=...
CP=$SCALA_HOME/lib/scala-library.jar:$SCALA_SPECS_HOME/specs-1.3.1.jar:bin
rm -rf bin
mkdir -p bin
scalac -d bin -cp $CP src/example/*.scala
scala -cp $CP example/CounterObserverSpec

Note that I put all the sources in a src/example directory. Also, I’m using v1.3.1 of specs, as well as v2.7.1 of Scala. You should get the following output.

    
Specification "CounterObserverSpec" 
  A Counter Observer should
  + observe counter increments

Total for specification "CounterObserverSpec":
Finished in 0 second, 60 ms
1 example, 2 assertions, 0 failure, 0 error

Observing with Aspects

Because Scala compiles to Java byte code, I can use AspectJ to advice Scala code! For this to work, you have to be aware of how Scala represents its concepts in byte code. For example, object declarations, e.g., object Foo {...} become static final classes. Also, method names like + become $plus in byte code.

However, most Scala type, method, and variable names can be used as is in AspectJ. This is true for my example.

Here is an aspect that observes calls to Counter.add.

    
package example

public aspect CounterObserver {
    after(Object counter, int value): 
        call(void *.add(int)) && target(counter) && args(value) {

        RecordedObservations.record("adding "+value);
    }
}

You can read this aspect as follows, after calling Counter.add (and keeping track of the Counter object that was called, and the value passed to the method), call the static method record on the RecordedObservations.

I’m using a separate Scala object RecordedObservations

    
package example

object RecordedObservations {
    private var messages = List[String]()
    def record(message: String):Unit = messages ::= message
    def count() = messages.length
    def reset():Unit = messages = Nil
}

Recall that this is effectively a static final Java class. I need this separate object, rather than keeping information in the aspect itself, because of the simple-minded way I’m building the code. ;) However, it’s generally a good idea with aspects to delegate most of the work to Java or Scala code anyway.

Now, the “spec” file is:

    
package example

import org.specs._

object CounterObserverSpec extends Specification {
    "A Counter Observer" should {
        "observe counter increments" in {
            RecordedObservations.reset()
            var counter = new Counter
            for (i <- 1 to 3) counter.add(i)
            RecordedObservations.count() must_== 3
            counter.count must_== 6
    }
  }
}

This time, I don’t need two more classes for the adding a mixin trait or defining an observer. Also, I call RecordedObservations.count to ensure it was called 3 times.

The build script is also slightly different to add the AspectJ compilation.

    
SCALA_HOME=...
SCALA_SPECS_HOME=...
ASPECTJ_HOME=...
CP=$SCALA_HOME/lib/scala-library.jar:$SCALA_SPECS_HOME/specs-1.3.1.jar:$ASPECTJ_HOME/lib/aspectjrt.jar:bin
rm -rf bin app.jar
mkdir -p bin
scalac -d bin -cp $CP src/example/*.scala 
ajc -1.5 -outjar app.jar -cp $CP -inpath bin src/example/CounterObserver.aj
aj -cp $ASPECTJ_HOME/lib/aspectjweaver.jar:app.jar:$CP example.CounterObserverSpec

The ajc command not only compiles the aspect, but it “weaves” into the compiled Scala classes in the bin directory. Actually, it only affects the Counter class. Then it writes all the woven and unmodified class files to app.jar, which is used to execute the test. Note that for production use, you might prefer load-time weaving.

The output is the same as before (except for the milliseconds), so I won’t show it here.

Comparing Traits with Aspects

So far, both approaches are equally viable. The traits approach obviously doesn’t require a separate language and corresponding tool set.

However, traits have one important limitation with respect to aspects. Aspects let you define pointcuts that are queries over all possible points where new behavior or modifications might be desired. These points are called join points in aspect terminology. The aspect I showed above has a simple pointcut that selects one join point, calls to the Counter.add method.

However, what if I wanted to observe all state changes in all classes in a package? Defining traits for each case would be tedious and error prone, since it would be easy to overlook some cases. With an aspect framework like AspectJ, I can implement observation at all the points I care about in a modular way.

Aspect frameworks support this by providing wildcard mechanisms. I won’t go into the details here, but the * in the previous aspect is an example, matching any type. Also, one of the most powerful techniques for writing robust aspects is to use pointcuts that reference only annotations, a form of abstraction. As a final example, if I add an annotation Adder to Counter.add,

    
package example

class Counter {
    var count = 0
    @Adder def add(i: Int) = count += i
}

Then I can rewrite the aspect as follows.

    
package example

public aspect CounterObserver {
    after(Object counter, int value): 
        call(@Adder void *.*(int)) && target(counter) && args(value) {

        RecordedObservations.record("adding "+value);
    }
}

Now, there are no type and method names in the pointcut. Any instance method on any visible type that takes one int (or Scala Int) argument and is annotated with Adder will get matched.

Note: Scala requires that you create any custom annotations as normal Java annotations. Also, if you intend to use them with Aspects, use runtime retention policy, which will be necessary if you use load-time weaving.

Conclusion

If you need to mix in behavior in a specific, relatively-localized set of classes, Scala traits are probably all you need and you don’t need another language. If you need more “pervasive” modifications (e.g., tracing, policy enforcement, security), consider using aspects.

Acknowledgements

Thanks to Ramnivas Laddad, whose forthcoming 2^nd Edition of AspectJ in Action got me thinking about this topic.

User Stories for Cross-Component Teams

Dean Wampler — Fri, 19 Sep 2008 19:53:00 -0500

I’m working on an Agile Transition for a large organization. They are organized into component teams. They implement features by forming temporary feature teams with representatives from each of the relevant components, usually one developer per component.

Doing User Stories for such cross-component features can be challenging.

Now, it would be nice if the developers just pair-programmed with each other, ignoring their assigned component boundaries, but we’re not quite there yet. Also, there are other issues we are addressing, such as the granularity of feature definitions, etc., etc. Becoming truly agile will take time.

Given where we are, it’s just not feasible to estimate a single story point value for each cross-component user story, because the work for each component varies considerably. A particular story might be the equivalent of 1 point for the UI part, 8 points for the middle-tier part, 2 points for the database part, etc.

So, what we’re doing is treating the user story as an “umbrella”, with individual component stories underneath. We’re estimating and tracking the points for each component story. The total points for the user story is the sum of the component story points, plus any extra we decide is needed for the integration and final acceptance testing work.

This model allows us to track the work more closely, as long as we remember that component points mean nothing from the point of view of delivering customer value!

I prefer this approach to documenting tasks, because it keeps the focus on delivering value to the client of each story. For the component stories, the client will be another component.

Automated Acceptance Tests for Component Stories

Just as for user stories, we are defining automated acceptance tests for each component. We’re using JUnit for them, since we don’t need a customer-friendly specification format, like FitNesse or RSpec.

This is also a (sneaky…) way to get the developers from different components to pair together. Say for example that we have a component story for the midtier and the UI is the client. The UI developer and the midtier developer pair to produce the the acceptance criteria for the story.

For each component story, the pair of programmers produce the following:

JUnit tests that define the acceptance criteria for the component story.
One or more interfaces that will be used by the client of the component. They will also be implemented by concrete classes in the component.
A test double that passes the JUnit tests and allows the client to move forward while the component feature is being implemented.

In a sense, the “contract” of the component story is the interfaces, which specify the static structure, and the JUnit tests, which specify the dynamic behavior of the feature.

This model of pair-programming the component interface should solve the common, inefficient communication problems when component interactions need to be changed. You know the scenario; a client component developer or manager tells a server component developer or manager that a change is needed. A developer (probably a different one…) on the server component team makes up an interface, checks it into version control, and waits for feedback from the client team. Meanwhile, the server component developer starts implementing the changes.

A few days before the big drop to QA for final integration testing, the server component developer realizes that the interface is missing some essential features. At the same time, the client component developer finally gets around to using the new interface and discovers a different set of missing essential features. Hilarity ensues…

We’re just getting started with this approach, but so far it is proving to be an effective way to organize our work and to be more efficient.

Configuration Management Systems, Automated Tests, CI, and Complexity

Dean Wampler — Mon, 08 Sep 2008 16:53:00 -0500

I’m working with a client that has a very complex branching structure in their commercial CM system, which will remain nameless. Why is it so complex? Because everyone is afraid to merge to the integration branches.

This is a common symptom in teams that don’t have have good automated test coverage and don’t use continuous integration (CI). Fear is their lot in life. They’ll keep lots of little branches and only merge to integration when they’re ready for the “big bang” integration.

I spoke with a manager at the client site today who expressed frustration that no one really knows which branch they should be committing to and when they should merge to the integration branches.

In contrast, projects with good test coverage and CI do almost all their work on the integration branch. They have little fear, because any problems will get caught and fixed quickly. So, the first point of this post is:

Automated test coverage and CI drastically simplify your use of configuration management.

Here’s something else I’ve noticed, open source CM systems seem to focus on different priorities than commercial systems. Disclaimer: I know I’m generalizing a bit here.

The commercial systems tend to be really good at managing complex branching, with fancy GUI tools to help manage the big trees of branches, facilitate merges, etc. That seems to be their biggest selling feature, GUI’s to manage the complexity.

In contrast, most of the open-source CM systems have lower-tech GUI’s, if any, but the teams using them don’t seem to care that much. Usually, this is because these teams are also practicing TDD and CI, so they just don’t need the wizardry as much.

The open-source CM systems seem to be better at scalability and performance. Some are pioneering distributed CM, e.g., Git and Mercurial. Git, for example, was designed to manage a massive project called Linux. Maybe you’ve heard of it.

Distributed CM is not easy, but it’s a lot easier to do if you don’t need to worry as much about complex branch hierarchies.

Most of the commercial tools I’ve seen don’t scale well and some require way too much administration. My client is apparently the biggest user of their particular tool and the developers complain all the time about performance. This tool is not designed to scale horizontally. The only hope is use faster hardware. In this case, the vendor has focused on managing complexity. To be frank, even their GUI tool is an uninspired and slow Java fat client.

So the second point of this post is:

Avoid CM tools that encourage complexity. Pick the ones that scale.

The Liskov Substitution Principle for "Duck-Typed" Languages

Dean Wampler — Sat, 06 Sep 2008 23:48:00 -0500

OCP and LSP together tell us how to organize similar vs. variant behaviors. I blogged the other day about OCP in the context of languages with open classes (i.e., dynamically-typed languages). Let’s look at the Liskov Substitution Principle (LSP).

The Liskov Substitution Principle was coined by Barbara Liskov in Data Abstraction and Hierarchy (1987).

If for each object o1 of type S there is an object o2 of type T such that for all programs P defined in terms of T, the behavior of P is unchanged when o1 is substituted for o2, then S is a subtype of T.

I’ve always liked the elegant simplicity, yet power, of LSP. In less formal terms, it says that if a client (program) expects objects of one type to behave in a certain way, then it’s only okay to substitute objects of another type if the same expectations are satisfied.

This is our best definition of inheritance. The well-known is-a relationship between types is not precise enough. Rather, the relationship has to be behaves-as-a, which unfortunately is more of a mouthful. Note that is-a focuses on the structural relationship, while behaves-as-a focuses on the behavioral relationship. A very useful, pre-TDD design technique called Design by Contract emerges out of LSP, but that’s another topic.

Note that there is a slight assumption that I made in the previous paragraph. I said that LSP defines inheritance. Why inheritance specifically and not substitutability, in general? Well, inheritance has been the main vehicle for substitutability for most OO languages, especially the statically-typed ones.

For example, a Java application might use a simple tracing abstraction like this.

    
public interface Tracing {
    void trace(String message);
}

Clients might use this to trace methods calls to a log. Only classes that implement the Tracer interface can be given to these clients. For example,

    
public class TracerClient {
    private Tracer tracer;

    public TracerClient(Tracer tracer) {
        this.tracer = tracer;
    }

    public void doWork() {
        tracer.trace("in doWork():");
        // ...
    }
}

However, Duck Typing is another form of substitutability that is commonly seen in dynamically-typed languages, like Ruby and Python.

If it walks like a duck and quacks like a duck, it must be a duck.

Informally, duck typing says that a client can use any object you give it as long as the object implements the methods the client wants to invoke on it. Put another way, the object must respond to the messages the client wants to send to it.

The object appears to be a “duck” as far as the client is concerned.

In or example, clients only care about the trace(message) method being supported. So, we might do the following in Ruby.

    
class TracerClient 
  def initialize tracer 
    @tracer = tracer
  end

  def do_work
    @tracer.trace "in do_work:" 
    # ... 
  end
end

class MyTracer
  def trace message
    p message
  end
end

client = TracerClient.new(MyTracer.new)

No “interface” is necessary. I just need to pass an object to TracerClient.initialize that responds to the trace message. Here, I defined a class for the purpose. You could also add the trace method to another type or object.

So, LSP is still essential, in the generic sense of valid substitutability, but it doesn’t have to be inheritance based.

Is Duck Typing good or bad? It largely comes down to your view about dynamically-typed vs. statically-typed languages. I don’t want to get into that debate here! However, I’ll make a few remarks.

On the negative side, without a Tracer abstraction, you have to rely on appropriate naming of objects to convey what they do (but you should be doing that anyway). Also, it’s harder to find all the “tracing-behaving” objects in the system.

On the other hand, the client really doesn’t care about a “Tracer” type, only a single method. So, we’ve decoupled “client” and “server” just a bit more. This decoupling is more evident when using closures to express behavior, e.g., for Enumerable methods. In our case, we could write the following.

    
class TracerClient2 
  def initialize &tracer 
    @tracer = tracer
  end

  def do_work 
    @tracer.call "in do_work:" 
    # ... 
  end
end

client = TracerClient2.new {|message| p "block tracer: #{message}"}

For comparison, consider how we might approach substitutability in Scala. As a statically-typed language, Scala doesn’t support duck typing per se, but it does support a very similar mechanism called structural types.

Essentially, structural types let us declare that a method parameter must support one or more methods, without having to say it supports a full interface. Loosely speaking, it’s like using an anonymous interface.

In our Java example, when we declare a tracer object in our client, we would be able to declare that is supports trace, without having to specify that it implements a full interface.

To be explicit, recall our Java constructor for TestClient.

    
public class TracerClient {
    public TracerClient(Tracer tracer) { ... }
    // ...
    }
}

In Scala, a complete example would be the following.

    
class ScalaTracerClient(val tracer: { def trace(message:String) }) {
    def doWork() = { tracer.trace("doWork") }
}

class ScalaTracer() {
    def trace(message: String) = { println("Scala: "+message) }
}

object TestScalaTracerClient {
    def main() {
        val client = new ScalaTracerClient(new ScalaTracer())
        client.doWork();
    }
}
TestScalaTracerClient.main()

Recall from my previous blogs on Scala, the argument list to the class name is the constructor arguments. The constructor takes a tracer argument whose “type” (after the ’:’) is { def trace(message:String) }. That is, all we require of tracer is that it support the trace method.

So, we get duck type-like behavior, but statically type checked. We’ll get a compile error, rather than a run-time error, if someone passes an object to the client that doesn’t respond to tracer.

To conclude, LSP can be reworded very slightly.

If for each object o1 of type S there is an object o2 of type T such that for all programs P defined in terms of T, the behavior of P is unchanged when o1 is substituted for o2, then S is substitutable for T.

I replaced a subtype of with substitutable for.

An important point is that the idea of a “contract” between the types and their clients is still important, even in a language with duck-typing or structural typing. However, languages with these features give us more ways to extend our system, while still supporting LSP.

The Open-Closed Principle for Languages with Open Classes

Dean Wampler — Thu, 04 Sep 2008 21:42:00 -0500

We’ve been having a discussion inside Object Mentor World Design Headquarters about the meaning of the OCP for dynamic languages, like Ruby, with open classes.

For example, in Ruby it’s normal to define a class or module, e.g.,

    
# foo.rb
class Foo
    def method1 *args
        ...
    end
end

and later re-open the class and add (or redefine) methods,

    
# foo2.rb
class Foo
    def method2 *args
        ...
    end
end

Users of Foo see all the methods, as if Foo had one definition.

    
foo = Foo.new
foo.method1 :arg1, :arg2
foo.method2 :arg1, :arg2

Do open classes violate the Open-Closed Principle? Bertrand Meyer articulated OCP. Here is his definition¹.

Software entities (classes, modules, functions, etc.) should be open for extension, but closed for modification.

He elaborated on it here.

... This is the open-closed principle, which in my opinion is one of the central innovations of object technology: the ability to use a software component as it is, while retaining the possibility of adding to it later through inheritance. Unlike the records or structures of other approaches, a class of object technology is both closed and open: closed because we can start using it for other components (its clients); open because we can at any time add new properties without invalidating its existing clients.

Tell Less, Say More: The Power of Implicitness

So, if one client require’s only foo.rb and only uses method1, that client doesn’t care what foo2.rb does. However, if the client also require’s foo2.rb, perhaps indirectly through another require, problems will ensue unless the client is unaffected by what foo2.rb does. This looks a lot like the way “good” inheritance should behave.

So, the answer is no, we aren’t violating OCP, as long as we extend a re-opened class following the same rules we would use when inheriting from it.

If we use inheritance instead:

    
# foo.rb
class Foo
    def method1 *args
        ...
    end
end
...
class DerivedFoo < Foo
    def method2 *args
        ...
    end
end
...
foo = SubFoo.new    # Instantiate different class...
foo.method1 :arg1, :arg2
foo.method2 :arg1, :arg2

One notable difference is that we have to instantiate a different class. This is an important difference. While you can often just use inheritance, and maybe you should prefer it, inheritance only works if you have full control over what types get instantiated and it’s easy to change which types you use. Of course, inheritance is also the best approach when you need all behavioral variants simulateneously, i.e., each variant in one or more objects.

Sometimes you want to affect the behavior of all instances transparently, without changing the types that are instantiated. A slightly better example, logging method calls, illustrates the point. Here we use the “famous” alias_method in Ruby.

    
# foo.rb
class Foo
    def method1 *args
        ...
    end
end
# logging_foo.rb
class Foo
    alias_method :old_method1, :method1
    def method1 *args
        p "Inside method1(#{args.inspect})" 
        old_method1 *args
    end
end
...
foo = Foo.new
foo.method1 :arg1, :arg2

Foo.method1 behaves like a subclass override, with extended behavior that still obeys the Liskov-Substitution Principle (LSP).

So, I think the OCP can be reworded slightly.

Software entities (classes, modules, functions, etc.) should be open for extension, but closed for source modification.

We should not re-open the original source, but adding functionality through a separate source file is okay.

Actually, I prefer a slightly different wording.

Software entities (classes, modules, functions, etc.) should be open for extension, but closed for source and contract modification.

The extra and contract is redundant with LSP. I don’t think this kind of redundancy is necessarily bad. ;) The contract is the set of behavioral expectations between the “entity” and its client(s). Just as it is bad to break the contract with inheritance, it is also bad to break it through open classes.

OCP and LSP together are our most important design principles for effective organization of similar vs. variant behaviors. Inheritance is one way we do this. Open classes provide another way. Aspects provide a third way and are subject to the same design issues.

¹ Meyer, Bertrand (1988). Object-Oriented Software Construction. Prentice Hall. ISBN 0136290493.

Tag: How Did I Get Started in Software Development

Dean Wampler — Fri, 29 Aug 2008 22:36:00 -0500

Micah Martin tagged me a while ago:

How old were you when you started programming.

I was around 15.

How did you get started programming.

This was in the mid-70’s. In school, we had access to a time-shared computer running Basic. My first “real” programs were in college when I wrote Fortran code for a PDP-11 in a Physics professor’s lab.

What was your first language?

Basic

What languages have you used since you started programming?

Roughly in order of adoption:

Fortran, C, Assembly Language, C++, PL/1, Perl, TCL/TK, Python, Java, HTML, JavaScript, CSS, SQL, Ruby, Scala

What was the first real program you wrote?

Data analysis programs in Fortran for that Physics professor. Later, in graduate school, I started applying OO principles to the massive Fortran simulations I wrote. I needed OO to manage the complex objects!

What was your first professional programming gig?

Writing PL/1 and C code for a 3-dimensional scanning system running on the RMX OS on proprietary Intel mini-computers. The tools were atrocious and we forced our customers to use a green screen terminal interface, because nothing else was available!

If there is one thing you learned along the way that you would tell new developers, what would it be?

Take the initiative to learn from potential mentors. They are too hard to find in our industry, so grab the opportunities when you can. The other recommendation I would make is to pay attention to the business side. Do you really want to do all that hard work for a project that will face-plant once it reaches the marketplace??

What’s the most fun you’ve ever had programming?

Leading a team of C++ developers writing a new user interface for a debugger that worked with in-circuit emulators (ICE’s). I always enjoyed UI development and the technical challenges were good. Unfortunately, it was one of those face-plants…

Tag, you’re it: Bob Koss, Brett Schuchert

The Seductions of Scala, Part III - Concurrent Programming

Dean Wampler — Thu, 14 Aug 2008 16:00:00 -0500

This is my third and last blog entry on The Seductions of Scala, where we’ll look at concurrency using Actors and draw some final conclusions.

Writing Robust, Concurrent Programs with Scala

The most commonly used model of concurrency in imperative languages (and databases) uses shared, mutable state with access synchronization. (Recall that synchronization isn’t necessary for reading immutable objects.)

However, it’s widely known that this kind of concurrency programming is very difficult to do properly and few programmers are skilled enough to write such programs.

Because pure functional languages have no side effects and no shared, mutable state, there is nothing to synchronize. This is the main reason for the resurgent interest in function programming recently, as a potential solution to the so-called multicore problem.

Instead, most functional languages, in particular, Erlang and Scala, use the Actor model of concurrency, where autonomous “objects” run in separate processes or threads and they pass messages back and forth to communicate. The simplicity of the Actor model makes it far easier to create robust programs. Erlang processes are so lightweight that it is common for server-side applications to have thousands of communicating processes.

Actors in Scala

Let’s finish our survey of Scala with an example using Scala’s Actors library.

Here’s a simple Actor that just counts to 10, printing each number, one per second.

    
import scala.actors._
object CountingActor extends Actor { 
    def act() { 
        for (i <- 1 to 10) { 
            println("Number: "+i)
            Thread.sleep(1000) 
        } 
    } 
} 

CountingActor.start()

The last line starts the actor, which implicitly invokes the act method. This actor does not respond to any messages from other actors.

Here is an actor that responds to messages, echoing the message it receives.

    
import scala.actors.Actor._ 
val echoActor = actor {
    while (true) {
        receive {
            case msg => println("received: "+msg)
        }
    }
}
echoActor ! "hello" 
echoActor ! "world!"

In this case, we do the equivalent of a Java “static import” of the methods on Actor, e.g., actor. Also, we don’t actually need a special class, we can just create an object with the desired behavior. This object has an infinite loop that effectively blocks while waiting for an incoming message. The receive method gets a block that is a match statement, which matches on anything received and prints it out.

Messages are sent using the target_actor ! message syntax.

As a final example, let’s do something non-trivial; a contrived network node monitor.

    
import scala.actors._
import scala.actors.Actor._
import java.net.InetAddress 
import java.io.IOException

case class NodeStatusRequest(address: InetAddress, respondTo: Actor) 

sealed abstract class NodeStatus
case class Available(address: InetAddress) extends NodeStatus
case class Unresponsive(address: InetAddress, reason: Option[String]) extends NodeStatus

object NetworkMonitor extends Actor {
    def act() {
        loop {
            react {  // Like receive, but uses thread polling for efficiency.
                case NodeStatusRequest(address, actor) => 
                    actor ! checkNodeStatus(address)
                case "EXIT" => exit()
            }
        }
    }
    val timeoutInMillis = 1000;
    def checkNodeStatus(address: InetAddress) = {
        try {
            if (address.isReachable(timeoutInMillis)) 
                Available(address)
            else
                Unresponsive(address, None)
        } catch {
            case ex: IOException => 
                Unresponsive(address, Some("IOException thrown: "+ex.getMessage()))
        }
    }
}

// Try it out:

val allOnes = Array(1, 1, 1, 1).map(_.toByte)
NetworkMonitor.start()
NetworkMonitor ! NodeStatusRequest(InetAddress.getByName("www.scala-lang.org"), self)
NetworkMonitor ! NodeStatusRequest(InetAddress.getByAddress("localhost", allOnes), self)
NetworkMonitor ! NodeStatusRequest(InetAddress.getByName("www.objectmentor.com"), self)
NetworkMonitor ! "EXIT" 
self ! "No one expects the Spanish Inquisition!!" 

def handleNodeStatusResponse(response: NodeStatus) = response match {
    // Sealed classes help here
    case Available(address) => 
        println("Node "+address+" is alive.")
    case Unresponsive(address, None) => 
        println("Node "+address+" is unavailable. Reason: <unknown>")
    case Unresponsive(address, Some(reason)) => 
        println("Node "+address+" is unavailable. Reason: "+reason)
}

for (i <- 1 to 4) self.receive {   // Sealed classes don't help here
    case (response: NodeStatus) => handleNodeStatusResponse(response)
    case unexpected => println("Unexpected response: "+unexpected)
}

We begin by importing the Actor classes, the methods on Actor, like actor, and a few Java classes we need.

Next we define a sealed abstract base class. The sealed keyword tells the compiler that the only subclasses will be defined in this file. This is useful for the case statements that use them. The compiler will know that it doesn’t have to worry about potential cases that aren’t covered, if new NodeStatus subclasses are created. Otherwise, we would have to add a default case clause (e.g., case _ => ...) to prevent warnings (and possible errors!) about not matching an input. Sealed class hierarchies are a useful feature for robustness (but watch for potential Open/Closed Principle violations!).

The sealed class hierarchy encapsulates all the possible node status values (somewhat contrived for the example). The node is either Available or Unresponsive. If Unresponsive, an optional reason message is returned.

Note that we only get the benefit of sealed classes here because we match on them in the handleNodeStatusResponse message, which requires a response argument of type NodeStatus. In contrast, the receive method effectively takes an Any argument, so sealed classes don’t help on the line with the comment “Sealed classes don’t help here”. In that case, we really need a default, the case unexpected => ... clause. (I added the message self ! "No one expects the Spanish Inquisition!!" to test this default handler.)

In the first draft of this blog post, I didn’t know these details about sealed classes. I used a simpler implementation that couldn’t benefit from sealed classes. Thanks to the first commenter, LaLit Pant, who corrected my mistake!

The NetworkMonitor loops, waiting for a NodeStatusRequest or the special string “EXIT”, which tells it to quit. Note that the actor sending the request passes itself, so the monitor can reply to it.

The checkNodeStatus attempts to contact the node, with a 1 second timeout. It returns an appropriate NodeStatus.

Then we try it out with three addresses. Note that we pass self as the requesting actor. This is an Actor wrapping the current thread, imported from Actor. It is analogous to Java’s Thread.currentThread().

Curiously enough, when I run this code, I get the following results.

    
Unexpected response: No one expects the Spanish Inquisition!!
Node www.scala-lang.org/128.178.154.102 is unavailable. Reason: <unknown>
Node localhost/1.1.1.1 is unavailable. Reason: <unknown>
Node www.objectmentor.com/206.191.6.12 is alive.

The message about the Spanish Inquisition was sent last, but processed first, probably because self sent it to itself.

I’m not sure why www.scala-lang.org couldn’t be reached. A longer timeout didn’t help. According to the Javadocs for InetAddress.isReachable), it uses ICMP ECHO REQUESTs if the privilege can be obtained, otherwise it tries to establish a TCP connection on port 7 (Echo) of the destination host. Perhaps neither is supported on the scala-lang.org site.

Conclusions

Here are some concluding observations about Scala vis-à-vis Java and other options.

A Better Java

Ignoring the functional programming aspects for a moment, I think Scala improves on Java in a number of very useful ways, including:

A more succinct syntax. There’s far less boilerplate, like for fields and their accessors. Type inference and optional semicolons, curly braces, etc. also reduce “noise”.
A true mixin model. The addition of traits solves the problem of not having a good DRY way to mix in additional functionality declared by Java interfaces.
More flexible method names and invocation syntax. Java took away operator overloading; Scala gives it back, as well as other benefits of using non-alphanumeric characters in method names. (Ruby programmers enjoy writing list.empty?, for example.)
Tuples. A personal favorite, I’ve always wanted the ability to return multiple values from a method, without having to create an ad hoc class to hold the values.
Better separation of mutable vs. immutable objects. While Java provides some ability to make objects final, Scala makes the distinction between mutability and immutability more explicit and encourages the latter as a more robust programming style.
First-class functions and closures. Okay, these last two points are really about FP, but they sure help in OO code, too!
Better mechanisms for avoiding null’s. The Option type makes code more robust than allowing null values.
Interoperability with Java libraries. Scala compiles to byte code so adding Scala code to existing Java applications is about as seamless as possible.

So, even if you don’t believe in FP, you will gain a lot just by using Scala as a better Java.

Functional Programming

But, you shouldn’t ignore the benefits of FP!

Better robustness. Not only for concurrent programs, but using immutable objects (a.k.a. value objects) reduces the potential for bugs.
A workable concurrency model. I use the term workable because so few developers can write robust concurrent code using the synchronization on shared state model. Even for those of you who can, why bother when Actors are so much easier??
Reduced code complexity. Functional code tends to be very succinct. I can’t overestimate the importance of rooting out all accidental complexity in your code base. Excess complexity is one of the most pervasive detriments to productivity and morale that I see in my clients’ code bases!
First-class functions and closures. Composition and succinct code are much easier with first-class functions.
Pattern matching. FP-style pattern matching makes “routing” of messages and delegation much easier.

Of course, you can mimic some of these features in pure Java and I encourage you to do so if you aren’t using Scala.

Static vs. Dynamic Typing

The debate on the relative merits of static vs. dynamic typing is outside our scope, but I will make a few personal observations.

I’ve been a dedicated Rubyist for a while. It is hard to deny the way that dynamic typing simplifies code and as I said in the previous section, I take code complexity very seriously.

Scala’s type system and type inference go a long way towards providing the benefits of static typing with the cleaner syntax of dynamic typing, but Scala doesn’t eliminate the extra complexity of static typing.

Recall my Observer example from the first blog post, where I used traits to implement it.

    
trait Observer[S] {
    def receiveUpdate(subject: S);
}

trait Subject[S] { 
    this: S =>
    private var observers: List[Observer[S]] = Nil
    def addObserver(observer: Observer[S]) = observers = observer :: observers

    def notifyObservers() = observers.foreach(_.receiveUpdate(this))
}

In Ruby, we might implement it this way.

    
module Subject 
    def add_observer(observer) 
      @observers ||= []
      @observers << observer  # append, rather than replace with new array
  end

    def notify_observers
      @observers.each {|o| o.receive_update(self)} if @observers
  end
end

There is no need for an Observer module. As long as every observer responds to the receive_update “message”, we’re fine.

I commented the line where I append to the existing @observers array, rather than build a new one, which would be the FP and Scala way. Appending to the existing array would be more typical of Ruby code, but this implementation is not as thread safe as an FP-style approach.

The trailing if expression in notify_observers means that nothing is done if @observers is still nil, i.e., it was never initialized in add_observer.

So, which is better? The amount of code is not that different, but it took me significantly longer to write the Scala version. In part, this was due to my novice chops, but the reason it took me so long was because I had to solve a design issue resulting from the static typing. I had to learn about the typed self construct used in the first line of the Subject trait. This was the only way to allow the Observer.receiveUpdate method accept to an argument of type S, rather than of type Subject[S]. It was worth it to me to achieve the “cleaner” API.

Okay, perhaps I’ll know this next time and spend about the same amount of time implementing a Ruby vs. Scala version of something. However, I think it’s notable that sometimes static typing can get in the way of your intentions and goal of achieving clarity. (At other times, the types add useful documentation.) I know this isn’t the only valid argument you can make, one way or the other, but it’s one reason that dynamic languages are so appealing.

Poly-paradigm Languages vs. Mixing Several Languages

So, you’re convinced that you should use FP sometimes and OOP sometimes. Should you pick a poly-paradigm language, like Scala? Or, should you combine several languages, each of which implements one paradigm?

A potential downside of Scala is that supporting different modularity paradigms, like OOP and FP, increases the complexity in the language. I think Odersky and company have done a superb job combining FP and OOP in Scala, but if you compare Scala FP code to Haskell or Erlang FP code, the latter tend to be more succinct and often easier to understand (once you learn the syntax).

Indeed, Scala will not be easy for developers to master. It will be a powerful tool for professionals. As a consultant, I work with developers with a range of skills. I would not expect some of them to prosper with Scala. Should that rule out the language? NO. Rather it would be better to “liberate” the better developers with a more powerful tool.

So, if your application needs OOP and FP concepts interspersed, consider Scala. If your application needs discrete services, some of which are predominantly OOP and others of which are predominantly FP, then consider Scala or Java for the OOP parts and Erlang or another FP language for the FP parts.

Also, Erlang’s Actor model is more mature than Scala’s, so Erlang might have an edge for a highly-concurrent server application.

Of course, you should do your own analysis…

Final Thoughts

Java the language has had a great ride. It was a godsend to us beleaguered C++ programmers in the mid ‘90’s. However, compared to Scala, Java now feels obsolete. The JVM is another story. It is arguably the best VM available.

I hope Scala replaces Java as the main JVM language for projects that prefer statically-typed languages. Fans of dynamically-typed languages might prefer JRuby, Groovy, or Jython. It’s hard to argue with all the OOP and FP goodness that Scala provides. You will learn a lot about good language and application design by learning Scala. It will certainly be a prominent tool in my toolkit from now on.

The Seductions of Scala, Part II - Functional Programming

Dean Wampler — Tue, 05 Aug 2008 20:32:00 -0500

A Functional Programming Language for the JVM

In my last blog post, I discussed Scala’s support for OOP and general improvements compared to Java. In this post, which I’m posting from Agile 2008, I discuss Scala’s support for functional programming (FP) and why it should be of interest to OO developers.

A Brief Overview of Functional Programming

You might ask, don’t most programming languages have functions? FP uses the term in the mathematical sense of the word. I hate to bring up bad memories, but you might recall from your school days that when you solved a function like

    
y = sin(x)

for y, given a value of x, you could input the same value of x an arbitrary number of times and you would get the same value of y. This means that sin(x) has no side effects. In other words, unlike our imperative OO or procedural code, no global or object state gets changed. All the work that a mathematical function does has to be returned in the result.

Similarly, the idea of a variable is a little different than what we’re used to in imperative code. While the value of y will vary with the value of x, once you have fixed x, you have also fixed y. The implication for FP is that “variables” are immutable; once assigned, they cannot be changed. I’ll call such immutable variables value objects.

Now, it would actually be hard for a “pure” FP language to have no side effects, ever. I/O would be rather difficult, for example, since the state of the input or output stream changes with each operation. So, in practice, all “pure” FP languages provide some mechanisms for breaking the rules in a controlled way.

Functions are first-class objects in FP. You can create named or anonymous functions (e.g., closures or blocks), assign them to variables, pass them as arguments to other functions, etc. Java doesn’t support this. You have to create objects that wrap the methods you want to invoke.

Functional programs tend to be much more declarative in nature than imperative programs. This is perhaps more obvious in pure FP languages, like Erlang and Haskell, than it is in Scala.

For example, the definition of Fibonacci numbers is the following.

    
F(n) = F(n-1) + F(n-2) where F(1)=1 and F(2)=1

An here is a complete implementation in Haskell.

    
module Main where 
-- Function f returns the n'th Fibonacci number. 
-- It uses binary recursion. 
f n | n <= 2 = 1 
    | n >  2 = f (n-1) + f (n-2)

Without understanding the intricacies of Haskell syntax, you can see that the code closely matches the “specification” above it. The f n | ... syntax defines the function f taking an argument n and the two cases of n values are shown on separate lines, where one case is for n <= 2 and the other case if for n > 2.

The code uses the recursive relationship between different values of the function and the special-case values when n = 1 and n = 2. The Haskell runtime does the rest of the work.

It’s interesting that most domain-specific languages are also declarative in nature. Think of how JMock, EasyMock or Rails’ ActiveRecord code look. The code is more succinct and it lets the “system” do most of the heavy lifting.

Functional Programming’s Benefits for You

Value Objects and Side-Effect Free Functions

It’s the immutable variables and side-effect free functions that help solve the multicore problem. Synchronized access to shared state is not required if there is no state to manage. This makes robust concurrent programs far easier to write.

I’ll discuss concurrency in Scala in my third post. For now, let’s discuss other ways that FP in Scala helps to improve code, concurrent or not.

Value objects are beneficial because you can pass one around without worrying that someone will change it in a way that breaks other users of the object. Value objects aren’t unique to FP, of course. They have been promoted in Domain Driven Design (DDD), for example.

Similarly, side-effect free functions are safer to use. There is less risk that a caller will change some state inappropriately. The caller doesn’t have to worry as much about calling a function. There are fewer surprises and everything of “consequence” that the function does is returned to the caller. It’s easier to keep to the Single Responsibility Principle when writing side-effect free functions.

Of course, you can write side-effect free methods and immutable variables in Java code, but it’s mostly a matter of discipline; the language doesn’t give you any enforcement mechanisms.

Scala gives you a helpful enforcement mechanism; the ability to declare variables as val’s (i.e., “values”) vs. var’s (i.e., “variables”, um… back to the imperative programming sense of the word…). In fact, val is the default, where neither is required by the language. Also, the Scala library contains both immutable and mutable collections and it “encourages” you to use the immutable collections.

However, because Scala combines both OOP and FP, it doesn’t force FP purity. The upside is that you get to use the approach that best fits the problem you’re trying to solve. It’s interesting that some of the Scala library classes expose FP-style interfaces, immutability and side-effect free functions, while using more traditional imperative code to implement them!

Closures and First-Class Functions

True to its functional side, Scala gives you true closures and first-class functions. If you’re a Groovy or Ruby programmer, you’re used to the following kind of code.

    
class ExpensiveResource {
    def open(worker: () => Unit) = {
        try {
            println("Doing expensive initialization")
            worker()
        } finally {
            close()
        }
    }
    def close() = {
        println("Doing expensive cleanup")
    }
}
// Example use:
try {
    (new ExpensiveResource()) open { () =>        // 1
        println("Using Resource")                 // 2
        throw new Exception("Thrown exception")   // 3
    }                                             // 4
} catch {
    case ex: Throwable => println("Exception caught: "+ex)
}

Running this code will yield:

    
Doing expensive initialization
Using Resource
Doing expensive cleanup
Exception caught: java.lang.Exception: Thrown exception

The ExpensiveResource.open method invokes the user-specified worker function. The syntax worker: () => Unit defines the worker parameter as a function that takes no arguments and returns nothing (recall that Unit is the equivalent of void).

ExpensiveResource.open handles the details of initializing the resource, invoking the worker, and doing the necessary cleanup.

The example marked with the comment // 1 creates a new ExpensiveResource, then calls open, passing it an anonymous function, called a function literal in Scala terminology. The function literal is of the form (arg_list_) => function body or () => println(...) ..., in our case.

A special syntax trick is used on this line; if a method takes one argument, you can change expressions of the form object.method(arg) to object method {arg}. This syntax is supported to allow user-defined methods to read like control structures (think for statements – see the next section). If you’re familiar with Ruby, the four commented lines read a lot like Ruby syntax for passing blocks to methods.

Idioms like this are very important. A library writer can encapsulate all complex, error-prone logic and allow the user to specify only the unique work required in a given situation. For example, How many times have you written code that opened an I/O stream or a database connection, used it, then cleaned up. How many times did you get the idiom wrong, especially the proper cleanup when an exception is thrown? First-class functions allow writers of I/O, database and other resource libraries to do the correct implementation once, eliminating user error and duplication. Here’s a rhetorical question I always ask myself:

How can I make it impossible for the user of this API to fail?

Iterations

Iteration through collections, Lists in particular, is even more common in FP than in imperative languages. Hence, iteration is highly evolved. Consider this example:

    
object RequireWordsStartingWithPrefix {
    def main(args: Array[String]) = {
        val prefix = args(0)
        for {
            i <- 1 to (args.length - 1)   // no semicolon
            if args(i).startsWith(prefix)
        } println("args("+i+"): "+args(i))
    }
}

Compiling this code with scalac and then running it on the command line with the command

    
scala RequireWordsStartingWithPrefix xx xy1 xx1 yy1 xx2 xy2

produces the result

    
args(2): xx1
args(5): xx2

The for loop assigns a loop variable i with each argument, but only if the if statement is true. Instead of curly braces, the for loop argument list could also be parenthesized, but then each line as shown would have to be separated by a semi-colon, like we’re used to seeing with Java for loops.

We can have an arbitrary number of assignments and conditionals. In fact, it’s quite common to filter lists:

    
object RequireWordsStartingWithPrefix2 {
    def main(args: Array[String]) = {
        val prefix = args(0)
        args.slice(1, args.length)
            .filter((arg: String) => arg.startsWith(prefix))
            .foreach((arg: String) => println("arg: "+arg))
    }
}

This version yields the same result. In this case, the args array is sliced (loping off the search prefix), the resulting array is filtered using a function literal and the filtered array is iterated over to print out the matching arguments, again using a function literal. This version of the algorithm should look familiar to Ruby programmers.

Rolling Your Own Function Objects

Scala still has to support the constraints of the JVM. As a comment to the first blog post said, the Scala compiler wraps closures and “bare” functions in Function objects. You can also make other objects behave like functions. If your object implements the apply method, that method will be invoked if you put parentheses with an matching argument list on the object, as in the following example.

    
class HelloFunction {
    def apply() = "hello" 
    def apply(name: String) = "hello "+name
}
val hello = new HelloFunction
println(hello())        // => "hello" 
println(hello("Dean"))  // => "hello Dean"

Option, None, Some…

Null pointer exceptions suck. You can still get them in Scala code, because Scala runs on the JVM and interoperates with Java libraries, but Scala offers a better way.

Typically, a reference might be null when there is nothing appropriate to assign to it. Following the conventions in some FP languages, Scala has an Option type with two subtypes, Some, which wraps a value, and None, which is used instead of null. The following example, which also demonstrates Scala’s Map support, shows these types in action.

    
val hotLangs = Map(
    "Scala" -> "Rocks", 
    "Haskell" -> "Ethereal", 
    "Java" -> null)
println(hotLangs.get("Scala"))          // => Some(Rocks)
println(hotLangs.get("Java"))           // => Some(null)
println(hotLangs.get("C++"))            // => None

Note that Map stores values in Options objects, as shown by the println statements.

By the way, those -> aren’t special operators; they’re methods. Like ::, valid method names aren’t limited to alphanumerics, _, and $.

Pattern Matching

The last FP feature I’ll discuss in this post is pattern matching, which is exploited more fully in FP languages than in imperative languages.

Using our previous definition of hotLangs, here’s how you might use matching.

    
def show(key: String) = {
    val value: Option[String] = hotLangs.get(key)
    value match {
        case Some(x) => x
        case None => "No hotness found" 
    }
}
println(show("Scala"))  // => "Rocks" 
println(show("Java"))   // => "null" 
println(show("C++"))    // => "No hotness found"

The first case statement, case Some(x) => x, says “if the value I’m matching against is a Some that could be constructed with the Some[+String](x: A) constructor, then return the x, the thing the Some contains.” Okay, there’s a lot going on here, so more background information is in order.

In Scala, like Ruby and other languages, the last value computed in a function is returned by it. Also, almost everything returns a value, including match statements, so when the Some(x) => x case is chosen, x is returned by the match and hence by the function.

Some is a generic class and the show function returns a String, so the match is to Some[+String]. The + in the +String expression is analogous to Java’s extends, i.e., <? extends String>. Capiche?

Idioms like case Some(x) => x are called extractors in Scala and are used a lot in Scala, as well as in FP, in general. Here’s another example using Lists and our friend ::, the “cons” operator.

    
def countScalas(list: List[String]): Int = {
    list match {
        case "Scala" :: tail => countScalas(tail) + 1
        case _ :: tail       => countScalas(tail)
        case Nil             => 0
    }
}
val langs = List("Scala", "Java", "C++", "Scala", "Python", "Ruby")
val count = countScalas(langs)
println(count)    // => 2

We’re counting the number of occurrences of “Scala” in a list of strings, using matching and recursion and no explicit iteration. An expression of the form head :: tail applied to a list returns the first element set as the head variable and the rest of the list set as the tail variable. In our case, the first case statement looks for the particular case where the head equals Scala. The second case matches all lists, except for the empty list (Nil). Since matches are eager, the first case will always pick out the List("Scala", ...) case first. Note that in the second case, we don’t actually care about the value, so we use the placeholder _. Both the first and second case’s call countScalas recursively.

Pattern matching like this is powerful, yet succinct and elegant. We’ll see more examples of matching in the next blog post on concurrency using message passing.

Recap of Scala’s Functional Programming

I’ve just touched the tip of the iceberg concerning functional programming (and I hope I got all the details right!). Hopefully, you can begin to see why we’ve overlooked FP for too long!

In my last post, I’ll wrap up with a look at Scala’s approach to concurrency, the Actor model of message passing.

The Seductions of Scala, Part I

Dean Wampler — Sun, 03 Aug 2008 15:30:00 -0500

(Update 12/23/2008: Thanks to Apostolos Syropoulos for pointing out an earlier reference for the concept of “traits”).

Because of all the recent hoo-ha about functional programming (e.g., as a “cure” for the multicore problem), I decided to cast aside my dysfunctional ways and learn one of the FP languages. The question was, which one?

My distinguished colleague, Michael Feathers, has been on a Haskell binge of late. Haskell is a pure functional language and is probably most interesting as the “flagship language” for academic exploration, rather than production use. (That was not meant as flame bait…) It’s hard to underestimate the influence Haskell has had on language design, including Java generics, .NET LINQ and F#, etc.

However, I decided to learn Scala first, because it is a JVM language that combines object-oriented and functional programming in one language. At ~13 years of age, Java is a bit dated. Scala has the potential of replacing Java as the principle language of the JVM, an extraordinary piece of engineering that is arguably now more valuable than the language itself. (Note: there is also a .NET version of Scala under development.)

Here are some of my observations, divided over three blog posts.

First, a few disclaimers. I am a Scala novice, so any flaws in my analysis reflect on me, not Scala! Also, this is by no means an exhaustive analysis of the pros and cons of Scala vs. other options. Start with the Scala website for more complete information.

A Better OOP Language

Scala works seamlessly with Java. You can invoke Java APIs, extend Java classes and implement Java interfaces. You can even invoke Scala code from Java, once you understand how certain “Scala-isms” are translated to Java constructs (javap is your friend). Scala syntax is more succinct and removes a lot of tedious boilerplate from Java code.

For example, the following Person class in Java:

    
class Person {
    private String firstName;
    private String lastName;
    private int    age;

    public Person(String firstName, String lastName, int age) {
        this.firstName = firstName;
        this.lastName  = lastName;
        this.age       = age;
    }

    public void setFirstName(String firstName) { this.firstName = firstName; }
    public void String getFirstName() { return this.firstName; }
    public void setLastName(String lastName) { this.lastName = lastName; }
    public void String getLastName() { return this.lastName; }
    public void setAge(int age) { this.age = age; }
    public void int getAge() { return this.age; }
}

can be written in Scala thusly:

    
class Person(var firstName: String, var lastName: String, var age: Int)

Yes, that’s it. The constructor is the argument list to the class, where each parameter is declared as a variable (var keyword). It automatically generates the equivalent of getter and setter methods, meaning they look like Ruby-style attribute accessors; the getter is foo instead of getFoo and the setter is foo = instead of setFoo. Actually, the setter function is really foo_=, but Scala lets you use the foo = sugar.

Lots of other well designed conventions allow the language to define almost everything as a method, yet support forms of syntactic sugar like the illusion of operator overloading, Ruby-like DSL’s, etc.

You also get fewer semicolons, no requirements tying package and class definitions to the file system structure, type inference, multi-valued returns (tuples), and a better type and generics model.

One of the biggest deficiencies of Java is the lack of a complete mixin model. Mixins are small, focused (think Single Responsibility Principle ...) bits of state and behavior that can be added to classes (or objects) to extend them as needed. In a language like C++, you can use multiple inheritance for mixins. Because Java only supports single inheritance and interfaces, which can’t have any state and behavior, implementing a mixin-based design has always required various hacks. Aspect-Oriented Programming is also one partial solution to this problem.

The most exciting OOP enhancement Scala brings is its support for Traits, a concept first described here and more recently discussed here. Traits support Mixins (and other design techniques) through composition rather than inheritance. You could think of traits as interfaces with implementations. They work a lot like Ruby modules.

Here is an example of the Observer Pattern written as traits, where they are used to monitor changes to a bank account balance. First, here are reusable Subject and Observer traits.

    
trait Observer[S] {
    def receiveUpdate(subject: S);
}

trait Subject[S] { 
    this: S =>
    private var observers: List[Observer[S]] = Nil
    def addObserver(observer: Observer[S]) = observers = observer :: observers

    def notifyObservers() = observers.foreach(_.receiveUpdate(this))
}

In Scala, generics are declared with square brackets, [...], rather than angled brackets, <...>. Method definitions begin with the def keyword. The Observer trait defines one abstract method, which is called by the Subject to notify the observer of changes. The Subject is passed to the Observer.

This trait looks exactly like a Java interface. In fact, that’s how traits are represented in Java byte code. If the trait has state and behavior, like Subject, the byte code representation involves additional elements.

The Subject trait is more complex. The strange line, this: S => , is called a self type declaration. It tells the compiler that whenever this is referenced in the trait, treat its type as S, rather than Subject[S]. Without this declaration, the call to receiveUpdate in the notifyObservers method would not compile, because it would attempt to pass a Subject[S] object, rather than a S object. The self type declaration solves this problem.

The next line creates a private list of observers, initialized to Nil, which is an empty list. Variable declarations are name: type. Why didn’t they follow Java conventions, i.e., type name? Because this syntax makes the code easier to parse when type inference is used, meaning where the explicit :type is omitted and inferred.

In fact, I’m using type inference for all the method declarations, because the compiler can figure out what each method returns, in my examples. In this case, they all return type Unit, the equivalent of Java’s void. (The name Unit is a common term in functional languages.)

The third line defines a method for adding a new observer to the list. Notice that concrete method definitions are of the form

    
def methodName(parameter: type, ...) = {
    method body
}

In this case, because there is only one line, I dispensed with the {...}. The equals sign before the body emphasizes the functional nature of scala, that all methods are objects, too. We’ll revisit this in a moment and in the next post.

The method body prepends the new observer object to the existing list. Actually, a new list is created. The :: operator, called “cons”, binds to the right. This “operator” is really a method call, which could actually be written like this, observers.::(observer).

Our final method in Subject is notifyObservers. It iterates through observers and invokes the block observer.receiveUpdate(this) on each observer. The _ evaluates to the current observer reference. For comparison, in Ruby, you would define this method like so:

    
def notifyObservers() 
    @observers.each { |o| o.receiveUpdate(self) }
end

Okay, let’s look at how you would actually use these traits. First, our “plain-old Scala object” (POSO) Account.

    
class Account(initialBalance: Double) {
    private var currentBalance = initialBalance
    def balance = currentBalance
    def deposit(amount: Double)  = currentBalance += amount
    def withdraw(amount: Double) = currentBalance -= amount
}

Hopefully, this is self explanatory, except for two things. First, recall that the whole class declaration is actually the constructor, which is why we have an initialBalance: Double parameter on Account. This looks strange to the Java-trained eye, but it actually works well and is another example of Scala’s economy. (You can define multiple constructors, but I won’t go into that here…).

Second, note that I omitted the parentheses when I defined the balance “getter” method. This supports the uniform access principle. Clients will simply call myAccount.balance, without parentheses and I could redefine balance to be a var or val and the client code would not have to change!

Next, a subclass that supports observation.

    
class ObservedAccount(initialBalance: Double) extends Account(initialBalance) with Subject[Account] {
    override def deposit(amount: Double) = {
        super.deposit(amount)
        notifyObservers()
    }
    override def withdraw(amount: Double) = {
        super.withdraw(amount)
        notifyObservers()
    }
}

The with keyword is how a trait is used, much the way that you implement an interface in Java, but now you don’t have to implement the interface’s methods. We’ve already done that.

Note that the expression, ObservedAccount(initialBalance: Double) extends Account(initialBalance), not only defines the (single) inheritance relationship, it also functions as the constructor’s call to super(initialBalance), so that Account is properly initialized.

Next, we have to override the deposit and withdraw methods, calling the parent methods and then invoking notifyObservers. Anytime you override a concrete method, scala requires the override keyword. This tells you unambiguously that you are overriding a method and the Scala compiler throws an error if you aren’t actually overriding a method, e.g., because of a typo. Hence, the keyword is much more reliable (and hence useful…) than Java’s @Override annotation.

Finally, here is an Observer that prints to stdout when the balance changes.

    
class AccountReporter extends Observer[Account] {
    def receiveUpdate(account: Account) =
        println("Observed balance change: "+account.balance)
}

Rather than use with, I just extend the Observer trait, because I don’t have another parent class.

Here’s some code to test what we’ve done.

    
def changingBalance(account: Account) = {
    println("==== Starting balance: " + account.balance)
    println("Depositing $10.0")
    account.deposit(10.0)
    println("new balance: " + account.balance)
    println("Withdrawing $5.60")
    account.withdraw(5.6)
    println("new balance: " + account.balance)
}

var a = new Account(0.0)
changingBalance(a)

var oa = new ObservedAccount(0.0)
changingBalance(oa)
oa.addObserver(new AccountReporter)
changingBalance(oa)

Which prints out:

    
==== Starting balance: 0.0
Depositing $10.0
new balance: 10.0
Withdrawing $5.60
new balance: 4.4
==== Starting balance: 0.0
Depositing $10.0
new balance: 10.0
Withdrawing $5.60
new balance: 4.4
==== Starting balance: 4.4
Depositing $10.0
Observed balance change: 14.4
new balance: 14.4
Withdrawing $5.60
Observed balance change: 8.8
new balance: 8.8

Note that we only observe the last transaction.

Download Scala and try it out. Put all this code in one observer.scala file, for example, and run the command:

    
scala observer.scala

But Wait, There’s More!

In the next post, I’ll look at Scala’s support for Functional Programming and why OO programmers should find it interesting. In the third post, I’ll look at the specific case of concurrent programming in Scala and make some concluding observations of the pros and cons of Scala.

For now, here are some references for more information.

The Scala website, for downloads, documentation, mailing lists, etc.
Ted Neward’s excellent multipart introduction to Scala at developerWorks.
The forthcoming Programming in Scala book.

Always <tt>close()</tt> in a <tt>finally</tt> block

Dean Wampler — Thu, 31 Jul 2008 00:12:00 -0500

Here’s one for my fellow Java programmers, but it’s really generally applicable.

When you call close() on I/O streams, readers, writers, network sockets, database connections, etc., it’s easy to forgot the most appropriate idiom. I just spent a few hours fixing some examples of misuse in otherwise very good Java code.

What’s wrong the following code?

    
public void writeContentToFile(String content, String fileName) throws Exception {
    File output = new File(fileName);
    OutputStreamWriter writer = new OutputStreamWriter(new FileOutputStream(output), "UTF-8");
    writer.write(content);
    writer.close();
}

It doesn’t look all that bad. It tells it’s story. It’s easy to understand.

However, it’s quite likely that you won’t get to the last line, which closes the writer, from time to time. File and network I/O errors are common. For example, what if you can’t actually write to the location specified by fileName? So, we have to be more defensive. We want to be sure we always clean up.

The correct idiom is to use a try … finally … block.

    
public void writeContentToFile(String content, String fileName) throws Exception {
    File output = new File(getFileSystemPath() + contentFilename);
    OutputStreamWriter writer = null;
    try {
        writer = new OutputStreamWriter(new FileOutputStream(output), "UTF-8");
        writer.write(content);
    } finally {
        if (writer != null)
            writer.close();
    }
}

Now, no matter what happens, the writer will be closed, if it’s not null, even if writing the output was unsuccessful.

Note that we don’t necessarily need a catch block, because in this case we’re willing to let any Exceptions propagate up the stack (notice the throws clause). A lot of developers don’t realize that there are times when you need a try block, but not necessarily a catch block. This is one of those times.

So, anytime you need to clean up or otherwise release resources, use a finally block to ensure that the clean up happens, no matter what.

The Ascendency of Dynamic X vs. Static X, where X = ...

Dean Wampler — Sat, 26 Jul 2008 21:48:00 -0500

I noticed a curious symmetry the other day. For several values of X, a dynamic approach has been gaining traction over a static approach, in some cases for several years.

X = Languages

The Ascendency of Dynamic Languages vs. Static Languages

This one is pretty obvious. It’s hard not to notice the resurgent interest in dynamically-typed languages, like Ruby, Python, Erlang, and even stalwarts like Lisp and Smalltalk.

There is a healthy debate about the relative merits of dynamic vs. static typing, but the “hotness” factor is undeniable.

X = Correctness Analysis

The Ascendency of Dynamic Correctness Analysis vs. Static Correctness Analysis

Analysis of code to prove correctness has been a research topic for years and the tools have become pretty good. If you’re in the Java world, tools like PMD and FindBugs find a lot of real and potential issues.

One thing none of these tools have ever been able to do is to analyze conformance of your code to your project’s requirements. I suppose you could probably build such tools using the same analysis techniques, but the cost would be too prohibitive for individual projects.

However, while analyzing the code statically is very hard, watching what the code actually does at runtime is more tractable and cost-effective, using automated tests.

Test-driving code results in a suite of unit, feature, and acceptance tests that do a good enough job, for most applications, of finding logic and requirements bugs. The way test-first development improves the design helps ensure correctness in the first place.

It’s worth emphasizing that automated tests exercise the code using representative data sets and scenarios, so they don’t constitute a proof of correctness. However, they are good enough for most applications.

X = Optimization

The Ascendency of Dynamic Optimization vs. Static Optimization

Perhaps the least well known of these X’s is optimization. Mature compilers like gcc have sophisticated optimizations based on static analysis of code (you can see where this is going…).

On the other hand, the javac compiler does not do a lot of optimizations. Rather, the JVM does.

The JVM watches the code execute and it performs optimizations the compiler could never do, like speculatively inlining polymorphic method calls, based on which types are actually having their methods invoked. The JVM puts in low-overhead guards to confirm that its assumptions are valid for each invocation. If not, the JVM de-optimizes the code.

The JVM can do this optimization because it sees how the code is really used at runtime, while the compiler has no idea when it looks at the code.

Just as for correctness analysis, static optimizations can only go so far. Dynamic optimizations simply bypass a lot of the difficulty and often yield better results.

Steve Yegge provided a nice overview recently of JVM optimizations, as part of a larger discussion on dynamic languages.

There are other dynamic vs. static things I could cite (think networking), but I’ll leave it at these three, for now.

Contracts and Integration Tests for Component Interfaces

Dean Wampler — Sun, 29 Jun 2008 21:54:00 -0500

I am mentoring a team that is transitioning to XP, the first team in a planned, corporate-wide transition. Recently we ran into miscommunication problems about an interface we are providing to another team.

The problems didn’t surface until a “big-bang” integration right before a major release, when it was too late to fix the problem. The feature was backed out of the release, as a result.

There are several lessons to take away from this experience and a few techniques for preventing these problems in the first place.

End-to-end automated integration tests are a well-established way of catching these problems early on. The team I’m mentoring has set up its own continuous-integration (CI) server and the team is getting pretty good at writing acceptance tests using FitNesse. However, these tests only cover the components provided by the team, not the true end-to-end user stories. So, they are imperfect as both acceptance tests and integration tests. Our longer-term goal is to automate true end-to-end acceptance and integration tests, across all components and services.

In this particular case, the other team is following a waterfall-style of development, with big design up front. Therefore, my team needed to give them an interface to design against, before we were ready to actually implement the service.

There are a couple of problems with this approach. First, the two teams should really “pair” to work out the interface and behavior across their components. As I said, we’re just starting to go Agile, but my goal is to have virtual feature teams, where members of the required component teams come together as needed to implement features. This would help prevent the miscommunication of one team defining an interface and sharing it with another team through documentation, etc. Getting people to communicate face-to-face and to write code together would minimize miscommunication.

Second, defining a service interface without the implementation is risky, because it’s very likely you will miss important details. The best way to work out the details of the interface is to test drive it in some way.

This suggests another technique I want to introduce to the team. When defining an interface for external consumption, don’t just deliver the “static” interface (source files, documentation, etc.), also deliver working Mock Objects that the other team can test against. You should develop these mocks as you test drive the interface, even if you aren’t yet working on the full implementation (for schedule or other reasons).

The mocks encapsulate and enforce the behavioral contract of the interface. Design by Contract is a very effective way of thinking about interface design and implementing automated enforcement of it. Test-driven development mostly serves the same practical function, but thinking in “contractual” terms brings clarity to tests that is often missing in many of the tests I see.

Many developers already use mocks for components that don’t exist yet and find that the mocks help them design the interfaces to those components, even while the mocks are being used to test clients of the components.

Of course, there is no guarantee that the mocks faithfully represent the actual behavior, but they will minimize surprises. Whether you have mocks or not, there is no substitute for running automated integration tests on real components as soon as possible.

Observations on Test-Driving User Interfaces

Dean Wampler — Sun, 22 Jun 2008 16:52:00 -0500

Test driving user interface development has always been a challenge. Recently, I’ve worked with two projects where most of the work has been on the user-interface components.

The first project is using Adobe Flex to create a rich interface. The team decided to adopt FunFX for acceptance testing. You write your tests in Ruby, typically using Test::Unit or RSpec.

FunFX places some constraints on your Flex application. You have to define the GUI objects in MXML, the XML-based file format for Flex applications, rather than ActionScript, and you need to add ids to all elements you want to reference.[1]

These are reasonable constraints and the first constraint promotes better quality, in fact. The MXML format is more succinct (despite the XML “noise”) and declarative than ActionScript code. This is almost always true of UI code in most languages (with notable exceptions…). Declarative vs. imperative code tends to improve quality because less code means fewer bugs, less to maintain, and it frees the implementor of the declarative “language” to pick the best implementation strategies, optimizations, etc. This characteristic is typical of Functional Languages and well-designed Domain Specific Languages, as well.

I don’t think you can underestimate the benefit of writing less code. I see too many teams whose problems would diminish considerably if they just got rid of duplication and learned to be concise.

The second project is a wiki-based application written in Java. To make deployment as simple as possible, the implementors avoided the Servlet API (no need to install Tomcat, etc.) and rolled their own web server and page rendering components. (I’m not sure I would have made these decisions myself, but I don’t think they are bad, either…)

The rendering components are object-oriented and use a number of design patterns, such as page factories with builder objects that reflect the “widgets” in the UI, HTML tags, etc. This approach makes the UI very testable with JUnit and FitNesse. In fact, the development process was a model of test-driven development.

However, the final result is flawed! It is much too difficult to change the look and feel of the application, which is essential for most UI’s, especially web UI’s. The project made the wrong tradeoffs; the design choices met the requirements of TDD very well, but they made maintenance and enhancement expensive and tedious. The application is now several years old and it has become dated, because of the expense of “refreshing” the look and feel.

What should have been done? These days, most dynamic web UI’s are built with templating engines, of which there are many in the most common programming languages. Pages defined in a templating engine are very declarative, except for the special tags where behavior is inserted. The pages are easy to change. It is mostly obvious where a particular visual element is generated, since most of the “tags” in the template look exactly like the tags in the rendered page. “Declarative” templates, like good DSL’s, can be read, understood, and even edited by the stakeholders, in this case the graphical designers.

But how do you test these page templates? When test-driving UI’s it is important to decide what to test and what not to test. The general rule for TDD is to test anything that can break. The corollary, especially relevant for UI’s, is don’t test anything when you don’t care if it changes.

It is usually the dynamic behavior of the UI that can break and should be tested. Templating engines provide special tags for inserting dynamic behavior in the underlying language (Java, Ruby, etc.). This is what you should test. It is usually best to keep the scripts in these tags as small as possible; the scripts just delegate to code, which can be test-driven in the usual way.

I see too many UI tests that compare long strings of HTML. These tests break whenever someone makes a minor look and feel or other inconsequential change. Part of the art of UI TDD is knowing how to test just what can break and nothing more. In the second project, incidental changes to the UI break tests that should be agnostic to such changes.

To conclude, keep your UI’s as declarative as you can. Only test the “declarations” (e.g., templates) in areas where they might break, meaning if it changes, it’s a bug. You’ll get the full benefits of TDD and the freedom to change the UI easily and frequently, as needed.

¹ Disclaimer: my information on FunFX is second hand, so I might not have the details exactly correct; see the FunFX documentation for details.

What We Can Learn from the Ojibwe Language

Dean Wampler — Sat, 03 May 2008 09:48:57 -0500

Ojibwe (sometimes spelled Ojibwa; the last syllable is pronounced “way”) is one of the few Native American languages that isn’t immediately threatened with extinction. It is spoken by about 10,000 people around the Great Lakes region. Brothers David and Anton Treuer are helping to keep it alive, as they discussed in a recent Fresh Air interview.

Ojibwe is a language that is optimized for an aboriginal people whose lives and livelihoods depend on an intimate awareness of their environment, especially the weather and water conditions. They have many nouns and verbs for fine gradations of rain, snow, ice conditions, the way water waves look and sound, etc. You would want this clarity of detail if you ventured out on a lake every day to fish for dinner.

In the past, speaking languages like Ojibwe was actively suppressed by the government, in an attempt to assimilate Native Americans. Today, the threat of extinction is more from the sheer ubiquity of English. I think there is another force at play, too. People living a modern, so-called “developed” lifestyle just don’t need to be so aware of their environment anymore. In fact, most of us are pretty “tone deaf” to the nuances of weather and water, which is sad in a way. We just don’t perceive the need for the richness of an Ojibwe to communicate what’s important to us, like sports trivia and fashion tips.

So, what does Ojibwe have to do with programming languages? Our language choices inform the way we frame problem solving and design. I was reminded of this recently while reading Ted Neward’s series of articles on Scala. Scala is a JVM language that provides first-class support for functional programming and object-oriented design refinements like traits, which provide mixin behavior.

While you can write Java-like code in Scala, Neward demonstrates how exploiting Scala features can result in very different code for many problems. The Scala examples are simpler, but sometimes that simplicity only becomes apparent after you grasp the underlying design principle in use, like closures or functional idioms.

One of the best pieces of advice in the Pragmatic Programmer is to learn a new language every year. You should pick a language that is very different from what you know already, not one that is fundamentally similar. Even if you won’t use that language in your professional work, understanding its principles, patterns, and idioms will inform your work in whatever languages you actually use.

For example, there is a lot of fretting these days about concurrent programming, given the rise of multi-core CPUs and multiprocessor computers. We know how to write concurrent programs in our most popular imperative languages, like Java and C++, but that knowledge is somewhat specialized and not widely known in the community. This is the main reason that functional programming is suddenly interesting again. It is inherently easier to write concurrent applications using side-effect-free code. I expect that we will fail to meet the concurrency challenge if we rely exclusively on the mechanisms in our imperative languages.

So, you could adopt a functional language for all or part of your concurrent application. Or, if you can’t use Scala (or Haskell or Erlang or …) you could at least apply functional idioms, like side-effect-free functions, avoidance of mutable objects, etc. in your current imperative language. However, even that won’t be an option unless you understand those principles in the first place.

Learning a new language is more than learning a new vocabulary. It’s even more than learning new design techniques. It’s also learning to see common things from a fresh perspective, with greater clarity.

Unit Testing C and C++ ... with Ruby and RSpec!

Dean Wampler — Mon, 04 Feb 2008 22:08:00 -0600

If you’re writing C/C++ code, it’s natural to write your unit tests in the same language (or use C++ for your C test code). All the well-known unit testing tools take this approach.

I think we can agree that neither language offers the best developer productivity among all the language choices out there. Most of us use either language because of perceived performance requirements, institutional and industry tradition, etc.

There’s growing interest, however, in mixing languages, tools, and paradigms to get the best tool for a particular job. <shameless-plug>I’m giving a talk March 7^th at SD West on this very topic, called Polyglot and Poly-Paradigm Programming </shameless-plug>.

So, why not use a more productive language for your C or C++ unit tests? You have more freedom in your development chores than what’s required for production. Why not use Ruby’s RSpec, a Behavior-Driven Development tool for acceptance and unit testing? Or, you could use Ruby’s version of JUnit, called Test::Unit. The hard part is integrating Ruby and C/C++. If you’ve been looking for an excuse to bring Ruby (or Tcl or Python or Java or…) into your C/C++ environment, starting with development tasks is usually the path of least resistance.

I did some experimenting over the last few days to integrate RSpec using SWIG (Simplified Wrapper and Interface Generator), a tool for bridging libraries written in C and C++ to other languages, like Ruby. The Ruby section of the SWIG manual was very helpful.

My Proof-of-Concept Code

Here is a zip file of my experiment: rspec_for_cpp.zip

This is far from a complete and working solution, but I think it shows promise. See the Current Limitations section below for details.

Unzip the file into a directory. I’ll assume you named it rspec_for_cpp. You need to have gmake, gcc, SWIG and Ruby installed, along with the RSpec “gem”. Right now, it only builds on OS X and Linux (at least the configurations on my machines running those OS’s – see the discussion below). To run the build, use the following commands:

    
        $ cd rspec_for_cpp/cpp
        $ make

You should see it finish with the lines

    
        ( cd ../spec; spec *_spec.rb )
        .........

        Finished in 0.0***** seconds

        9 examples, 0 failures

Congratulations, you’ve just tested some C and C++ code with RSpec! (Or, if you didn’t succeed, see the notes in the Makefile and the discussion below.)

The Details

I’ll briefly walk you through the files in the zip and the key steps required to make it all work.

`cexample.h`

Here is a simple C header file.

    
        /* cexample.h */
        #ifndef CEXAMPLE_H
        #define CEXAMPLE_H
        #ifdef __cplusplus
         extern "C" {
        #endif
        char* returnString(char* input);
        double returnDouble(int input);
        void  doNothing();

        #ifdef __cplusplus
         }
        #endif
        #endif

Of course, in a pure C shop, you won’t need the #ifdef __cplusplus stuff. I found this was essential in my experiment when I mixed C and C++, as you might expect.

`cpp/cexample.c`

Here is the corresponding C source file.

    
        /* cexample.h */

        char* returnString(char* input) {
            return input;
        }

        double returnDouble(int input) {
            return (double) input;
        }

        void  doNothing() {}

`cpp/CppExample.h`

Here is a C++ header file.

    
        #ifndef CPPEXAMPLE_H
        #define CPPEXAMPLE_H

        #include <string>

        class CppExample 
        {
        public:
            CppExample();
            CppExample(const CppExample& foo);
            CppExample(const char* title, int flag);
            virtual ~CppExample();

            const char* title() const;
            void        title(const char* title);
            int         flag() const;
            void        flag(int value);

            static int countOfCppExamples();
        private:
            std::string _title;
            int         _flag;
        };

        #endif

`cpp/CppExample.cpp`

Here is the corresponding C++ source file.

    
        #include "CppExample.h" 

        CppExample::CppExample() : _title("") {}
        CppExample::CppExample(const CppExample& foo): _title(foo._title) {}
        CppExample::CppExample(const char* title, int flag) : _title(title), _flag(flag) {}
        CppExample::~CppExample() {}

        const char* CppExample::title() const { return _title.c_str(); }
        void        CppExample::title(const char* name) { _title = name; }

        int  CppExample::flag() const { return _flag; }
        void CppExample::flag(int value) { _flag = value; }

        int CppExample::countOfCppExamples() { return 1; }

`cpp/example.i`

Typically in SWIG, you specify a .i file to the swig command to define the module that wraps the classes and global functions, which classes and functions to expose to the target language (usually all in our case), and other assorted customization options, which are discussed in the SWIG manual. I’ll show the swig command in a minute. For now, note that I’m going to generate an example_wrap.cpp file that will function as the bridge between the languages.

Here’s my example.i, where I named the module example.

    
        %module example
        %{
            #include "cexample.h" 
            #include "CppExample.h"    
        %}
        %include "cexample.h" 
        %include "CppExample.h"

It looks odd to have header files appear twice. The code inside the %{...%} (with a ’#’ before each include) are standard C and C++ statements, etc. that will be inserted verbatim into the generated “wrapper” file, example_wrap.cpp, so that file will compile when it references anything declared in the header files. The second case, with a ’%’ before the include statements¹, tells SWIG to make all the declarations in those header files available to the target language. (You can be more selective, if you prefer…)

Following Ruby conventions, the Ruby plugin for SWIG automatically names the module with an upper case first letter (Example), but you use require 'example' to include it, as we’ll see shortly.

Building

See the cpp/Makefile for the gory details. In a nutshell, you run the swig command like this.

    
        swig -c++ -ruby -Wall -o example_wrap.cpp example.i

Next, you create a dynamically-linked library, as appropriate for your platform, so the Ruby interpreter can load the module dynamically when required. The Makefile can do this for Linux and OS X platforms. See the Ruby section of the SWIG manual for Windows specifics.

If you test-drive your code, which tends to drive you towards minimally-coupled “modules”, then you can keep your libraries and build times small, which will make the build and test cycle very fast!

`spec/cexample_spec.rb` and `spec/cppexample_spec.rb`

Finally, here are the RSpec files that exercise the C and C++ code. (Disclaimer: these aren’t the best spec files I’ve ever written. For one thing, they don’t exercise all the CppExample methods! So sue me… :)

    
        require File.dirname(__FILE__) + '/spec_helper'
        require 'example'

        describe "Example (C functions)" do
          it "should be a constant on Module" do
            Module.constants.should include('Example')
          end
          it "should have the methods defined in the C header file" do
            Example.methods.should include('returnString')
            Example.methods.should include('returnDouble')
            Example.methods.should include('doNothing')
          end
        end

        describe Example, ".returnString" do
          it "should return the input char * string as a Ruby string unchanged" do
            Example.returnString("bar!").should == "bar!" 
          end  
        end

        describe Example, ".returnDouble" do
          it "should return the input integer as a double" do
            Example.returnDouble(10).should == 10.0
          end
        end

        describe Example, ".doNothing" do
          it "should exist, but do nothing" do
            lambda { Example.doNothing }.should_not raise_error
          end
        end

and

    
    require File.dirname(__FILE__) + '/spec_helper'
    require 'example'

    describe Example::CppExample do
      it "should be a constant on module Example" do
        Example.constants.should include('CppExample')
      end
    end

    describe Example::CppExample, ".new" do
      it "should create a new object of type CppExample" do
        example = Example::CppExample.new("example1", 1)
        example.title.should == "example1" 
        example.flag.should  == 1
      end
    end

    describe Example::CppExample, "#title(value)" do
      it "should set the title" do
        example = Example::CppExample.new("example1", 1)
        example.title("title2")
        example.title.should == "title2" 
      end
    end

    describe Example::CppExample, "#flag(value)" do
      it "should set the flag" do
        example = Example::CppExample.new("example1", 1)
        example.flag(2)
        example.flag.should == 2
      end
    end

If you love RSpec like I do, this is a very compelling thing to see!

And now for the small print:

Current Limitations

As I said, this is just an experiment at this point. Volunteers to make this battle-ready would be most welcome!

General

The Example `Makefile` File

It Must Be Hand Edited for Each New or Renamed Source File

You’ve probably already solved this problem for your own make files. Just merge in the example Makefile to pick up the SWIG- and RSpec-related targets and rules.

It Only Knows How to Build Shared Libraries for Mac OS X and Linux (and Not Very Well)

Some definitions are probably unique to my OS X and Linux machines. Windows is not supported at all. However, this is also easy rectify. Start with the notes in the Makefile itself.

The `module.i` File Must Be Hand Edited for Each File Change

Since the format is simple, a make task could fill a template file with the changed list of source files during the build.

Better Automation

It should be straightforward to provide scripts, IDE/Editor shortcuts, etc. that automate some of the tasks of adding new methods and classes to your C and C++ code when you introduce them first in your “spec” files. (The true TDD way, of course.)

Specific Issues for C Code Testing

I don’t know of any other C-specific issues, so unit testing with Ruby is most viable today for C code. Although I haven’t experimented extensively, C functions and variables are easily mapped by SWIG to a Ruby module. The Ruby section of the SWIG manual discusses this mapping in some detail.

Specific Issues for C++ Code Testing

More work will be required to make this viable. It’s important to note that SWIG cannot handle all C++ constructs (although there are workarounds for most issues, if you’re committed to this approach…). For example, namespaces, nested classes, some template and some method overloading scenarios are not supported. The SWIG manual has details.

Also, during my experiment, SWIG didn’t seem to map const std::string& objects in C++ method signatures to Ruby strings, as I would have expected (char* worked fine).

Is It a Viable Approach?

Once the General issues listed above are handled, I think this approach would work very well for C code. For C++ code, there are more issues that need to be addressed, and programmers who are committed to this strategy will need to tolerate some issues (or just use C++-language tools for some scenarios).

Conclusions: Making It Development-Team Ready

I’d like to see this approach pushed to its logical limit. I think it has the potential to really improve the productivity of C and C++ developers and the quality of their test coverage, by leveraging the productivity and power of dynamically-typed languages like Ruby. If you prefer, you could use Tcl, Python, even Java instead.

License

This code is complete open and free to use. Of course, use it at your own risk; I offer it without warranty, etc., etc. When I polish it to the point of making it an “official” project, I will probably release under the Apache license.

¹ I spent a lot of time debugging problems because I had a ’#’ where I should have had a ’%’! Caveat emptor!

TDD for AspectJ Aspects

Dean Wampler — Tue, 02 Oct 2007 11:34:24 -0500

There was a query on the TDD mailing list about how to test drive aspects. Here is an edited version of my reply to that list.

Just as for regular classes, TDD can drive aspects to a better design.

Assume that I’m testing a logging aspect that logs when certain methods are called. Here’s the JUnit 4 test:

package logging;
import static org.junit.Assert.*;
import org.junit.Test;
import app.TestApp;

public class LoggerTest {
    @Test
    public void FakeLoggerShouldBeCalledForAllMethodsOnTestClasses() {
        String message = "hello!";
        new TestApp().doFirst(message);
        assertTrue(FakeLogger.messageReceived().contains(message));
        String message2 = "World!";
        new TestApp().doSecond(message, message2);
        assertTrue(FakeLogger.messageReceived().contains(message));
    }
}

Already, you might guess that FakeLogger will be a test-only version of something, in this case, my logging aspect. Similarly, TestApp is a simple class that I’m using only for testing. You might choose to use one or more production classes, though.

package app;
@Watchable
public class TestApp {
    public void doFirst(String message) {}
    public void doSecond(String message1, String message2) {}
}

and @Watchable is a marker annotation that allows me to define pointcuts in my logger aspect without fragile coupling to concrete names, etc. You could also use an interface.

package app;
public @interface Watchable {}

I made up @Watchable as a way of marking classes where the public methods might be of “interest” to particular observers of some kind. It’s analogous to the EJB 3 annotations that mark classes as “persistable” without implying too many details of what that might mean.

Now, the actual logging is divided into an abstract base aspect and a test-only concrete sub-aspect>

package logging;

import org.aspectj.lang.JoinPoint;
import app.Watchable;

abstract public aspect AbstractLogger {
    // Limit the scope to the packages and types you care about.
    public abstract pointcut scope();

    // Define how messages are actually logged.
    public abstract void logMessage(String message);

    // Notice the coupling is to the @Watchable abstraction.
    pointcut watch(Object object):
        scope() && call(* (@Watchable *).*(..)) && target(object);

    before(Object watchable): watch(watchable) {
        logMessage(makeLogMessage(thisJoinPoint));
    }

    public static String makeLogMessage(JoinPoint joinPoint) {
        StringBuffer buff = new StringBuffer();
        buff.append(joinPoint.toString()).append(", args = ");
        for (Object arg: joinPoint.getArgs())
            buff.append(arg.toString()).append(", ");
        return buff.toString();
    }
}

and

package logging;

public aspect FakeLogger extends AbstractLogger {
    // Only match on calls from the unit tests.
    public pointcut scope(): within(logging.*Test);

    public void logMessage(String message) {
        lastMessage = message; 
    }

    static String lastMessage = null;
    public static String messageReceived() {
        return lastMessage;
    }
}

Pointcuts in aspects are like most other dependencies, best avoided ;) ... or at least minimized and based on abstractions, just like associations and inheritance relationships.

So, my test “pressure” drove the design in terms of where I needed abstraction in the Logger aspect: (i) how a message is actually logged and (ii) what classes get “advised” with logging behavior.

Just as for TDD of regular classes, the design ends up with minimized dependencies and flexibility (abstraction) where it’s most useful.

I can now implement the real, concrete logger, which will also be a sub-aspect of AbstractLogger. It will define the scope() pointcut to be a larger section of the system and it will send the message to the real logging subsystem.

Why you have time for TDD (but may not know it yet...)

Dean Wampler — Sun, 30 Sep 2007 20:01:02 -0500

Note: Updated 9/30/2007 to improve the graphs and to clarify the content.

A common objection to TDD is this; “We don’t have time to write so many tests. We don’t even have enough time to write features!”

Here’s why people who say this probably already have enough time in the (real) schedule, they just don’t know it yet.

Let’s start with an idealized Scrum-style “burn-down chart” for a fictional project run in a “traditional” way (even though traditional projects don’t use burn-down charts…).

We have time increasing on the x axis and the number of “features” remaining to implement on the y axis (it could also be hours or “story points” remaining). During a project, a nice feature of burn-down charts is that you can extend the line to see where it intersects the x axis, which is a rough indicator of when you’ll actually finish.

The optimistic planners for our fictional project plan to give the software to QA near the end of the project. They expect QA to find nothing serious, so the release will occur soon thereafter on date T0.

Of course, it never works out that way:

The red line is the actual effort for our fictional project. It’s quite natural for the planned list of features to change as the team reacts to market changes, etc.. This is why the line goes up sometimes (in “good” projects, too!). Since this is a “traditional” project, I’m assuming that there are no automated tests that actually prove that a given feature is really done. We’re effectively running “open loop”, without the feedback of tests.

Inevitably, the project goes over budget and th planned QA drop comes late. Then things get ugly. Without our automated unit tests, there are lots of little bugs in the code. Without our automated integration tests, there are problems when the subsystems are run together. Without our acceptance tests, the implemented features don’t quite match the actual requirements for them.

Hence, a chaotic, end-of-project “birthing” period ensues, where QA reports a list of big and small problems, followed by a frantic effort (usually involving weekends…) by the developers to address the problems, followed by another QA drop, followed by…, and so forth.

Finally, out of exhaustion and because everyone else is angry at the painful schedule slip, the team declares “victory” and ships it, at time T1.

We’ve all lived through projects like this one.

Now, if you remember your calculus classes (sorry to bring up painful memories), you will recall that the area under the curve is the total quantity of whatever the curve represents. So, the actual total feature work required for our project corresponds to the area under the red line, while the planned work corresponds to the area under the black line. So, we really did have more time than we originally thought.

Now consider a Test-Driven Development (TDD) project [1]:

Here, the blue line is similar to the red line, at least early in the project. Now we have frequent “milestones” where we verify the state of the project with the three kinds of automated tests mentioned above. Each milestone is the end of an iteration (usually 1-4 weeks apart). Not shown are the 5-minute TDD cycles and the feedback from the continuous integration process that does our builds and runs all our tests after every block of commits to version control (many times a day).

The graph suggests that the total amount of effort will be higher than the expected effort without tests, which may be true [2]. However, because of the constant feedback during the whole life of the project, we really know where we actually are at any time. By measuring our progress in this way, we will know early whether or not we can meet the target date with the planned feature set. With early warnings, we can adjust accordingly, either dropping features or moving the target date, with relatively little pain. Whereas, without this feedback, we really don’t know what’s done until something, e.g., the QA process, gives us that feedback. Hence, at time T0, just before the big QA drop, the traditional project has little certainty about what features are really completed.

So, we’ll experience less of the traditional end-of-project chaos, because there will be fewer surprises. Without the feedback from automated tests, QA is find lots of problems, causing the chaotic and painful end-of-project experience. Finding and trying to fix major problems late in the game can even kill a project.

So, TDD converts that unknown schedule time at the end into known time early in the project. You really do have time for automated tests and your tests will make your projects more predictable and less painful at the end.

Note: I appreciate the early comments and questions that helped me clarify this post.

[1] As one commenter remarked, this post doesn’t actually make the case for TDD itself vs. alternative “test-heavy” strategies, but I think it’s pretty clear that TDD is the best of the known test-heavy strategies, as argued elsewhere.

[2] There is some evidence that TDD and pair programming lead to smaller applications, because they help avoid unnecessary features. Also, they provide constant feedback to the team, including the stake holders, on what the feature set should really be and which features are most important to complete first.

Why you have time for TDD (but may not know it yet...)

Dean Wampler — Sun, 30 Sep 2007 20:01:02 -0500

Note: Updated 9/30/2007 to improve the graphs and to clarify the content.

A common objection to TDD is this; “We don’t have time to write so many tests. We don’t even have enough time to write features!”

Here’s why people who say this probably already have enough time in the (real) schedule, they just don’t know it yet.

Let’s start with an idealized Scrum-style “burn-down chart” for a fictional project run in a “traditional” way (even though traditional projects don’t use burn-down charts…).

Of course, it never works out that way:

Finally, out of exhaustion and because everyone else is angry at the painful schedule slip, the team declares “victory” and ships it, at time T1.

We’ve all lived through projects like this one.

Now consider a Test-Driven Development (TDD) project [1]:

Note: I appreciate the early comments and questions that helped me clarify this post.

Why you have time for TDD (but may not know it yet...)

Dean Wampler — Sun, 30 Sep 2007 20:01:02 -0500

Note: Updated 9/30/2007 to improve the graphs and to clarify the content.

A common objection to TDD is this; “We don’t have time to write so many tests. We don’t even have enough time to write features!”

Here’s why people who say this probably already have enough time in the (real) schedule, they just don’t know it yet.

Let’s start with an idealized Scrum-style “burn-down chart” for a fictional project run in a “traditional” way (even though traditional projects don’t use burn-down charts…).

Of course, it never works out that way:

Finally, out of exhaustion and because everyone else is angry at the painful schedule slip, the team declares “victory” and ships it, at time T1.

We’ve all lived through projects like this one.

Now consider a Test-Driven Development (TDD) project [1]:

Note: I appreciate the early comments and questions that helped me clarify this post.

ANN: Talks at the Chicago Ruby Users Group (Oct. 1, Dec. 3)

Dean Wampler — Sat, 29 Sep 2007 09:52:00 -0500

I’m speaking on Aquarium this Monday night (Oct. 1st). Details are here. David Chelimsky will also be talking about new developments in RSpec and RBehave.

At the Dec. 3 Chirb meeting, the two of us are reprising our Agile 2007 tutorial Ruby’s Secret Sauce: Metaprogramming.

Please join us!

ANN: OOPSLA Tutorial on "Principles of Aspect-Oriented Design in Java and AspectJ"

Dean Wampler — Thu, 13 Sep 2007 11:34:29 -0500

I’m doing a tutorial on aspect-oriented design principles with examples in Java and AspectJ at OOPSLA this year (October 21st). You can find a description here. I believe Friday, 9/14, is the last day for early, discounted registration, so sign up now!

A short presentation on the same subject can be found here.

Why we write code and don't just draw diagrams

Dean Wampler — Thu, 06 Sep 2007 10:45:59 -0500

Advocates of graphical notations have long hoped we would reach the point were we only draw diagrams and don’t write textual code. There have even been a few visual programming environments that have come and gone over the years.

If a picture is worth a thousand words, then why hasn’t this happened?

What that phrase really means is that we get the “gist” or the “gestalt” of a situation when we look at a picture, but nothing expresses the intricate details like text, the 1000 words. Since computers are literal-minded and don’t “do gist”, they require those details spelled out explicitly.

Well, couldn’t we still do that with a sufficiently expressive graphical notation? Certainly, but then we run into the pragmatic issue that typing textual details will always be faster than drawing them.

I came to this realization a few years ago when I worked for a Well Known Company developing UML-based tools for Java developers. The tool’s UI could have been more efficient, but there was no way to beat the speed of typing text.

It’s also true that some languages are rather verbose. This is one of the ways in which Domain-Specific Languages (DSL’s) are going to be increasingly important. A well-designed DSL will let you express those high-level concepts succinctly.

I’m not claiming that there is no place for graphical representations. UML is great for those quick design sessions, when you’re strategizing at a high level. Also, the easiest way to find component dependency cycles is to see them graphically.

I’m also not discounting those scenarios where a diagram-driven approach actually works. I’ve heard of some success developing control systems that are predominantly well-defined, complex state machines.

Still, for the general case, code written in succinct languages with well-designed API’s and DSL’s will trump a diagram-driven approach.

CJUG West 9/6/07: Aspect-Oriented Programming and Software Design

Dean Wampler — Tue, 04 Sep 2007 17:55:47 -0500

I’m giving a talk at the Chicago Java User’s Group West meeting this Thursday at 6:30 PM. The topic is Aspect-Oriented Programming and Software Design in Java and AspectJ. I’ll briefly describe the problems that AOP addresses and how the principles of object-oriented design influence AOP and vice versa. If you’re in the area, I hope to see you there.

Announcement: Aquarium v0.1.0 - An Aspect-Oriented Programming Toolkit for Ruby

Dean Wampler — Thu, 23 Aug 2007 17:26:00 -0500

I just released the first version of a new Aspect-Oriented Programming toolkit for Ruby called Aquarium. I blogged about the goals of the project here. Briefly, they are

Create a powerful pointcut language, the most important part of an AOP toolkit.
Provide Robust support for concurrent advice at the same join point..
Provide runtime addition and removal of aspects.
Provide a test bed for implementation ideas for DSL's.

There is extensive documentation at the Aquarium site. Please give it a try and let me know what you think!

Applications Should Use Several Languages

Dean Wampler — Wed, 04 Jul 2007 11:38:31 -0500

Yesterday, I blogged about TDD in C++ and ended with a suggestion for the dilemma of needing optimal performance some of the time and optimal productivity the rest of the time. I suggested that you should use more than one language for your applications.

If you are developing web applications, you are already doing this, of course. Your web tier probably uses several “languages”, e.g., HTML, JavaScript, JSP/ASP, CSS, Java, etc.

However, most people use only one language for the business/mid tier. I think you should consider using several; a high-productivity language environment for most of your work, with the occasional critical functionality implemented in C or C++ to optimize performance, but only after actually measuring where the bottlenecks are located.

This approach is much too rare, but it has historical precedents. One of the most successful and long-lived software projects of all time is Emacs. It consists of a core C-based runtime with most of the functionality implemented in Emacs lisp “components”. The relative ease of extending Emacs using lisp has resulted in a rich assortment of support tools for various operating systems, languages, build tools, etc. Even modern IDEs and and other graphical editors have not completely displaced Emacs.

Java has embraced the mixed language philosophy somewhat reluctantly. JNI is the official and most commonly-used API for invoking “native” code, but it is somewhat hard to use and few people actually use it. In contrast, for example, the Ruby world has always embraced this approach. Ruby has an easy to use API for invoking native C code and good alternatives exist for invoking code in other languages. As a result, many of the 3rd-party Ruby libraries (or gems) contain both Ruby and native C code. The latter is built on the fly when you install the gem. Hence, there are many high-performance Ruby applications. This is not a contradiction in terms, because the performance-critical sections run natively, even though interpreted Ruby is relatively slow.

Of course, you have to be judicious in how you use mixed-language programming. Crossing the language boundary is often somewhat heavyweight, so you should avoid doing such invocations inside tight loops, for example.

So, I think the solution to the dilemma of needing high performance sometimes and high productivity the rest of the time is to pick the right tools for each circumstance and make them interoperate. Even constrained embedded devices like cell phones would be easier to implement if most of the code were written in a language like Ruby, Python, Smalltalk, or Java and performance-critical components were written in C or C++.

If I were starting such a greenfield project, I would assume that time-to-money is the top priority and write most of my code in Ruby (my personal current favorite), using TDD of course. I would profile it constantly, as part of the nightly or continuous-integration build. When bottlenecks emerge, I would first determine if a refactoring is sufficient to fix them and if not, I would rewrite the critical sections in C. If the project were for an embedded device, I would also watch the resource usage carefully.

For my embedded device, I would test from the beginning whether or not the overhead of the interpreter/VM and the overall performance are acceptable. I would also be sure that I have adequate tool support for the inevitable remote debugging and diagnostics I’ll have to do. If I made the wrong tool choices after all, I would know early on, when it’s still relatively painless to retool.

If you’re an IT or web-site developer, you have fewer performance limitations and more options. You might decide to make the cross-language boundary a cross-process boundary, e.g., by communicating through some sort of lightweight web services. This is one way to leverage legacy C/C++ code while developing new functionality in a more productive language.

Observations on TDD in C++ (long)

Dean Wampler — Tue, 03 Jul 2007 23:15:09 -0500

I spent all of June mentoring teams on TDD in C++ with some Java. While C++ was my language of choice through most of the 90’s, I think far too many teams are using it today when there are better options for their particular needs.

During the month, I took notes on all the ways that C++ development is less productive than development in languages like Java, particular if you try to practice TDD. I’m not trying to start a language flame war. There are times when C++ is the appropriate tool, as we’ll see.

Most of the points below have been discussed before, but it is useful to list them in one place and to highlight a few particular observations.

Based on my observations last month, as well as previously experience, I’ve come to the conclusion that TDD in C++ is about an order of magnitude slower than TDD in Java. Mostly, this is due to poor or non-existent tool support for automated refactorings, no error detection as you type, and the requirement to compile and link an executable test.

So, here is my list of impediments that I encountered last month. I’ll mostly use Java as the comparison language, but the arguments are more or less the same for C# and the popular dynamic languages, like Ruby, Python, and Smalltalk. Note that the dynamic languages tend to have less complete tool support, but they make up for it in other ways (off-topic for this blog).

Getting Started

There is more setup effort involved in configuring your build environment to use your chosen unit testing framework (e.g., CppUnit) and to create small executables, one each for a single or a few tests. Creating many small tests, rather than one big test (e.g., a variant of the actual application). This is important to minimize the TDD cycle.

Fortunately, this setup is a one-time “charge”. The harder part, if you have legacy code, is refactoring it to break hard dependencies so you can write unit tests. This is true for legacy code in any language, of course.

Complex Syntax

C++ has a very complex syntax. This makes it hard to parse, limiting the capabilities of automated tools and slowing build times (more below).

The syntax also makes it harder to program in the language and not just for novices. Even for experts, the visual noise of pointer and reference syntax obscures the story the code is trying to tell. That is, C++ code is inherently less clean than code in most other languages in widespread use.

Also, the need for the developer to remember whether each variable is a pointer, a reference, or a “value”, and how to manage its life-cycle, requires mental effort that could be applied to the logic of the code instead.

Obsolete Tool Support

No editor or IDE supports non-trivial, automated refactorings. (Some do simple refactorings like “rename”.) This means you have to resort to tedious, slow, and error-prone manual refactorings. Extract Method is made worse by the fact that you usually have to edit two files, an implementation and a header file.

There are no widely-used tools that provide on-the-fly parsing and error indications. This alone increases the time between typing an error and learning about it by an order of magnitude. Since a build is usually required, you tend to type a lot between builds, thereby learning about many errors at once. Working through them takes time. (There may be some commercial tools with limited support for on-the-fly parsing, but they are not widely used.)

Similarly, none of the common development tools support incremental loading of object code that could be used for faster unit testing and hence a faster TDD cycle. Most teams just build executables. Even when they structure the build process to generate small, focused executables for unit tests, the TDD cycle times remain much longer than for Java.

Finally, while there is at least one mocking framework available for C++, it is much harder to use than comparable frameworks in newer languages.

Manual Memory Management

We all know that manual memory management leads to time spent finding and fixing memory errors and leaks. Avoiding these problems in the first place also consumes a lot of thought and design effort. In Java, you just spend far less time thinking about “who owns this object and is therefore responsible for managing its life-cycle”.

Dependency Management

Intelligent handling of include directives is entirely up to the developer. We have all used the following “guard” idiom:

    #ifndef MY_CLASS_H
    #define MY_CLASS_H
    ...
    #endif

Unfortunately, this isn’t good enough. The file will still get opened and read in its entirety every time it is included. You could also put the guard directives around the include statement:

    #ifndef MY_CLASS_H
    #include "myclass.h"
    #endif

This is tedious and few people do it, but it does avoid the wasted file I/O.

Finally, too few people simply declare a required class with no body:

    class MyClass;

This is sufficient when one header references another class as a pointer or reference. In our experience with clients, we have often seen build times improve significantly when teams cleaned up their header file usage and dependencies, in general. Still, why is all this necessary in the 21st century?

This problem is made worse by the unfortunate inclusion of private and protected declarations in the same header file included by clients of the class. This creates phantom dependencies from the clients to class details that they can’t access directly.

Other Debugging Issues

Limited or non-existent context information when an exception is thrown makes the origin of the exception harder to find. To fill the gap, you tend to spend more time adding this information manually through logging statements in catch blocks, etc.

The std::exception class doesn’t appear to have a std::string or const char* argument in a constructor for a message. You could just throw a string, but that precludes using an exception class with a meaningful name.

Compiler error messages are hard to read and often misleading. In part this is due to the complexity of the syntax and the parsing problem mentioned previously. Errors involving template usage are particular hard to debug.

Reflection and Metaprogramming

Many of the productivity gains from using dynamic languages and (to a lesser extent) Java and C# are due to their reflection and metaprogramming facilities. C++ relies more on template metaprogramming, rather than APIs or other built-in language features that are easier to use and more full-featured. Preprocessor hacks are also used frequently. Better reflection and metaprogramming support would permit more robust proxy or aspect solutions to be used. (However, to be fair, sometimes a preprocessor hack has the virtue of being “the simplest thing that could possibly work.”)

Library Issues

Speaking of std::string and char*, it is hard to avoid writing two versions of methods, one which takes const std::string& arguments and one which takes const char* arguments. It doesn’t matter that one method can usually delegate to the other one; this is wasted effort.

Discussion

So, C++ makes it hard for me to work the way that I want to work today, which is test-driven, creating clean code that works. That’s why I rarely choose it for a project.

However, to be fair, there are legitimate reasons for almost all of the perceived “deficiencies” listed above. C++ emphasizes performance and backwards-compatibility with C over all other considerations. However, they come at the expense of other interests, like effective TDD.

It is a good thing that we have languages that were designed with performance as the top design goal, because there are circumstances where performance is the number one requirement. However, most teams that use C++ as their primary language are making an optimal choice for, say, 10% of their code, but which is suboptimal the other 90%. Your numbers will vary; I picked 10% vs. 90% based on the fact that performance bottlenecks are usually localized and they should be found by actual measurements, not guesses!

Workarounds

If it’s true that TDD is an order of magnitude slower for C++ then what do we do? No doubt really good C++ developers have optimized their processes as best as they can, but in the end, you will just have to live with longer TDD cycles. Instead of write just enough test to fail, make it pass, refactor, it will be more like write a complete test, write the implementation, build it, fix the compilation errors, run it, fix the logic errors to make the test pass, and then refactor.

A Real Resolution?

You could consider switching to the D language, which is link compatible with C and appears to avoid many of the problems described above.

There is another way out of the dilemma of needing optimal performance some of the time and optimal productivity the rest of the time; use more than one language. I’ll discuss this idea in my next blog.

Observations on TDD in C++ (long)

Dean Wampler — Tue, 03 Jul 2007 23:15:09 -0500

Most of the points below have been discussed before, but it is useful to list them in one place and to highlight a few particular observations.

Getting Started

Complex Syntax

C++ has a very complex syntax. This makes it hard to parse, limiting the capabilities of automated tools and slowing build times (more below).

Obsolete Tool Support

Finally, while there is at least one mocking framework available for C++, it is much harder to use than comparable frameworks in newer languages.

Manual Memory Management

Dependency Management

Intelligent handling of include directives is entirely up to the developer. We have all used the following “guard” idiom:

    #ifndef MY_CLASS_H
    #define MY_CLASS_H
    ...
    #endif

Unfortunately, this isn’t good enough. The file will still get opened and read in its entirety every time it is included. You could also put the guard directives around the include statement:

    #ifndef MY_CLASS_H
    #include "myclass.h"
    #endif

This is tedious and few people do it, but it does avoid the wasted file I/O.

Finally, too few people simply declare a required class with no body:

    class MyClass;

Other Debugging Issues

Reflection and Metaprogramming

Library Issues

Discussion

So, C++ makes it hard for me to work the way that I want to work today, which is test-driven, creating clean code that works. That’s why I rarely choose it for a project.

Workarounds

A Real Resolution?

You could consider switching to the D language, which is link compatible with C and appears to avoid many of the problems described above.

Are "else" blocks the root of all evil?

Dean Wampler — Tue, 05 Jun 2007 14:42:00 -0500

So, I’m pair programming C++ code with a client today and he makes an observation that makes me pause.

The well-structured, open-source code I’ve looked at typically has very few else blocks. You might see a conditional test with a return statement if the conditional evaluates to true, but not many if/else blocks.

(I’m quoting from memory…) Now, this may seem crazy at first, but one of the principles we teach at Object Mentor is the Single Responsibility Principle, which states that a class should have only one reason to change. This principle also applies to methods. More loosely defined, a class or method should do only one thing.

So, if a method has an if/else block, is it doing two (or more) things and therefore violating the SRP?

Okay, so this is a bit too restrictive (and the title was a bit of an attention grabber… ;). We’re not talking about something really evil, like premature optimization, after all!

However, look at your own if/else blocks and ask yourself if maybe your code would express its intent better if you refactored it to eliminate some of those else blocks.

So, is there something to this idea?

What I've Learned from Master Chef Rino Baglio

Dean Wampler — Sun, 27 May 2007 21:39:00 -0500

If you want to learn Craftsmanship, you would be hard pressed to find a better mentor than Rino Baglio, the Executive Master Chef at Pazzaluna, in St. Paul, Minnesota. I had a chance to catch up with Rino this past weekend when I was in Minneapolis to teach a tutorial on Aspect-Oriented Design at ICSE 2007.

I have known Rino for over a decade, starting when Ann and I were loyal patrons of Il Bacio in Redmond, Washington, the restaurant he owned and operated with his wife Patsy until a few years ago. Through his cooking classes and many conversations about food and the restaurant business, I learned a lot about what it really means to be a chef and the long mentoring process that true chefs go through.

In the U.S., we think that passing a two-year culinary program qualifies you to be a chef. In Italy, an aspiring chef apprentices to a master at the age of 13 or so and spends the next 20-odd years mastering the craft before deserving the designation of “chef”. You can spend 7 years just working through the stations in a restaurant, cold dishes, salads, sauces, etc., just to become a “cook”.

Here are some of the characteristics of a true craftsman.

A craftsman is widely recognized by peers

Rino recently won an international competition in Italy, one of many times he’s been recognized nationally and internationally.

A craftsman is passionate about the craft

Rino says that if you are passionate about food, you will work on the presentation of even humble dishes. Pasta, as well as lobster, deserves an attractive presentation.

A craftsman delivers value to the customer while meeting business objectives

Rino keeps the kitchen lean and efficient. He keeps costs low by relying on high-quality ingredients, keeping waste to a minimum, and constantly improving the skills of his staff, all without ever compromising quality. In the year Rino has been at Pazzaluna, costs have dropped, while business and profits have increased.

A craftsman knows that quality is the number one priority

Rino knows that cutting quality today means less business tomorrow. He keeps quality high by keeping morale high. Morale is high because his staff is constantly learning new and better recipes. Also, as you watch him interact with his staff, you can see that he treats all of them, from his sous chefs to the dishwashers, with dignity and respect, while always holding them to high standards.

A craftsman never stops learning

You would think that he knows it all, by now. Yet, he has never forgotten a lesson his own mentor taught him, “you can learn something from even the worst cook, because he always knows something you don’t!” How many gurus do you know that think they have nothing left to learn?

What does all this have to do with software? Pretty much everything. Like cuisine, clean code is part art, part science. Clean code is created by passionate craftsman who are fanatical and fastidious about every detail. Clean code is the product of years of accumulated experience. The decisions a master makes moment-by-moment, whether test-driving the next feature or fighting a fire, reflect the wisdom and breadth of knowledge that produce high-quality results quickly and efficiently. Finally, a master leads by example, bringing the rest of the team up to his or her standards.

So, if you’re young and ambitious, latch onto the mentors around you. If you can’t find any, find another job. (Your organization is doomed anyway; so you might as well move on now.) If you’re older and wiser, seek out the promising junior people, teach them what you know, and learn from them as well! Oh, and if you want to taste real Italian food, make a pilgrimage to St. Paul. Tell Rino I sent you.

100% Code Coverage?

Dean Wampler — Wed, 16 May 2007 22:50:00 -0500

Should you strive for 100% code coverage from your unit tests? It’s probably not mandatory and if your goal is 100% coverage, you’ll focus on that goal and not focus on writing the best tests for the behavior of your code.

That said, here are some thoughts on why I still like to get close to 100%.

I’m anal retentive. I don’t like those little red bits in my coverage report. (Okay, that’s not a good reason…)
Every time I run the coverage report, I have to inspect all the uninteresting cases to find the interesting cases I should cover.
The tests are the specification and documentation of the code, so if something nontrivial but unexpected happens, there should still be a test to “document” the behavior, even if the test is hard to write.
Maybe those places without coverage are telling me to fix the design.

I was thinking about this last point the other day when considering a bit of Java code that does a downcast (assume that’s a good idea, for the sake of argument…), wrapped in a try/catch block for the potential ClassCastException:

public void handleEvent (Event event) throws ApplicationException {
  try {
    SpecialEvent specialEvent = (SpecialEvent) event;
    doSomethingSpecial (specialEvent);
  } catch (ClassCastException cce) { 
    throw new ApplicationException(cce);
  }
}

To get 100% coverage, you would have to write a test that inputs an object of a different subtype of Event to trigger coverage of the catch block. As we all know, these sorts of error-handling code blocks are typically the hardest to cover and ones we’re most likely to ignore. (When was the last time you saw a ClassCastException anyway?)

So my thought was this, we want 100% of the production code to be developed with TDD, so what if we made 100% coverage a similar goal? How would that change our designs? We might decide that since we have to write a test to cover this error-handling scenario, maybe we should rethink the scenario itself. Is it necessary? Could we eliminate the catch block with a better overall design, in this case, making sure that we test all callers and ensure that they obey the method’s ‘contract’? Should we just let the ClassCastException fly out of the function and let a higher-level catch block handle it? After all, catching and rethrowing a different exception is slightly smelly and the code would be cleaner without the try/catch block. (For completeness, a good use of exception wrapping is to avoid namespace pollution. We might not want application layer A to know anything about layer C’s exception types, so we wrap a C exception in an A exception, which gets passed through layer B…)

100% coverage is often impossible or impractical, because of language or tool oddities. Still, if you give in early, you’re overlooking some potential benefits.

AOP and Dynamic Languages: Contradiction in Terms or Match Made in Heaven?

Dean Wampler — Wed, 21 Mar 2007 13:21:35 -0500

Consider this quote from Dave Thomas (of the Pragmatic Programmers) on AOP (aspect-oriented programming, a.k.a. aspect-oriented software development - AOSD) :

Once you have decent reflection and metaclasses, AOP is just part of the language.

People who work with dynamic languages don't see any need for AOP-specific facilities in their language. They don't necessarily dispute the value of AOP for Java, where metaprogramming facilities are weaker, but for them, AOP is just a constrained form of metaprogramming. Are they right?

It's easy to see why people feel this way when you consider that most of the applications of AOP have been to solve obvious "cross-cutting concerns" (CCCs) like object-relational mapping, transactions, security, etc. In other words, AOP looks like just one of many tools in your toolbox to solve a particular group of problems.

I'll use Ruby as my example dynamic language, since Ruby is the example I know best. It's interesting to look at Ruby on Rails source code, where you find a lot of "AOP-like" code that addresses the CCCs I just mentioned (and more). This is easy enough to do using Ruby's metaprogramming tools, even though tooling that supports AOP semantics would probably make this code easier to write and maintain.

This is going to be a long blog entry already, so I won't cite detailed examples here, but consider how Rails uses method_missing to effectively "introduce" new methods into classes and modules. For example, in ActiveRecord, the many possible find methods and attribute read/write methods are "implemented" this way.

By the way, another excellent Ruby framework, RSpec used method_missing for similar purposes, but recently refactored its implementation and public API to avoid method_missing, because having multiple frameworks attempt to use the same "hook" proved very fragile!

Also in Rails, method "aliasing" is done approximately 175 times, often to wrap ("advise") methods with new behaviors.

Still, is there really a need for AOP tooling in dynamic languages? First, consider that in the early days of OOP, some of us "faked" OOP using whatever constructs our languages provided. I wrote plenty of C code that used structs as objects and method pointers to simulate method overloading and overriding. However, few people would argue today that such an approach is "good enough". If we're thinking in objects, it sure helps to have a language that matches those semantics.

Similarly, it's true that you can implement AOP using sufficiently powerful metaprogramming facilities, but it's a lot harder than having native AOP semantics in your language (or at least a close approximation thereof in libraries and their DSLs).

Before proceeding, let me remind you what AOP is for in the first place. AOP is essentially a new approach to modularization that complements other approaches, like objects. It tries to solve a group of problems that other modularity approaches can't handle, namely the fine-grained interaction of multiple domain models that is required to implement required functionality.

Take the classic example of security management. Presumably, you have one strategy and implementation for handling authentication and authorization. This is one domain and your application's "core business logic" is another domain.

In a non-AOP system, it is necessary to insert duplicate or nearly duplicate code throughout the system that invokes the security subsystem. This duplication violates DRY, it clutters the logic of the code where it is inserted, it is difficult to test, maintain, replace with a new implementation, etc.

Now you may say that you handle this through a Spring XML configuration file or an EJB deployment configuration file, for example. Congratulations, you are using an AOP or AOP-like system!

What AOP seeks to do is to allow you to specify that repeated behavior in one place, in a modular way.

There are four pieces required for an AOP system:

1. Interception

You need to be able to "intercept" execution points in the program. We call these join points in the AOP jargon and sets of them that the aspect writer wants to work with at once are called pointcuts (yes, no whitespace). At each join point, advice is the executable code that an aspect invokes either before, after or both before and after ("around") the join point.

Note that the most powerful AOP language, AspectJ, let's you advise join points like instance variable reads and writes, class initialization, instance creation, etc. The easiest join points to advise are method calls and many AOP systems limit themselves to this capability.

2. Introduction

Introduction is the ability to add new state and behavior to an existing class, object, etc. For example, if you want to use the Observer pattern with a particular class, you could use an aspect to introduce the logic to maintain the list of observers and to notify them when state changes occur.

3. Inspection

We need to be able to find the join points of interest, either through static or runtime analysis, preferably both! You would also like to specify certain conditions of interest, which I'll discuss shortly.

4. Modularization

If we can't package all this into a "module", then we don't have a new modularization scheme. Note that a part of this modularization is the ability to somehow specify in one place the behavior I want and have it affect the entire system. Hence, AOP is a modularity system with nonlocal effects.

Okay. How does pure Ruby stack up these requirements? If you're a Java programmer, the idea of Interception and Introduction, where you add new state and behavior to a class, may seem radical. In languages with "open classes" like Ruby, it is trivial and common to reopen a class (or Module) and insert new attributes (state) and methods (behavior). You can even change previously defined methods. Hence, Interception and Introduction are trivial in Ruby.

This is why Ruby programmers assume that AOP is nothing special, but what they are missing are the complete picture for Inspection and Modularization, even though both are partially covered.

There is a rich reflection API for finding classes and objects. You can write straightforward code that searches for classes that "respond to" a particular method, for example. What you can't do easily is query based on state. For example, in AspectJ, you can say, "I want to advise method call X.m when it is called in the context flow ('cflow') of method call Y.m2 somewhere up the stack..." Yes, you can figure out how to do this in Ruby, but it's hard. So, we're back to the argument I made earlier that you would really like your language to match the semantics of your ideas.

For modularization, yes you can put all the aspect-like code in a Module or Class. The hard part is encapsulating any complicated "point cut" metaprogramming in one place, should you want to use it again later. That is, once you figure out how to do the cflow pointcuts using metaprogramming, you'll want that tricky bit of code in a library somewhere.

At this point, you might be saying to yourself, "Okay, so it might be nice to have some AOP stuff in Ruby, but the Rails guys seem to be doing okay without it. Is it really worth the trouble having AOP in the language?" Only if AOP is more applicable than for the limited set of problems described previously.

Future Applications of AOP??

Here's what I've been thinking about lately. Ruby is a wonderful language for creating mini-DSLs. The ActiveRecord DSL is a good example. It provides relational semantics, while the library minimizes the coding required by the developer. (AR reads the database schema and builds an in-memory representation of the records as objects.)

Similarly, there is a lot of emphasis these days on development that centers around the domain or features of the project. Recall that I said that AOP is about modularizing the intersection of multiple domains (and recall my previous blog on the AOSD 2007 Conference where Gerald Sussman remarked that successful systems have more than one organizing principle).

I think we'll see AOP become the underlying implementation of powerful DSLs that allow programmers who are not AOP-literate express cross-cutting concerns in domain-specific and intuitive languages. AOP will do the heavy lifting behind the scenes to make the fine-grained interactions work. I really don't expect a majority of developers to become AOP literate any time soon. In my experience, too many so-called developers don't get objects. They'll never master aspects!

Shameless Plug

If you would like to hear more of my ideas about AOP in Ruby and aspect-oriented design (AOD), please come to my talk at SD West, this Friday at 3:30. I'm also giving a full-day tutorial on AOD in Ruby and Java/AspectJ at the ICSE 2007 conference in Minneapolis, May 26th.

AOSD 2007 Conference

Dean Wampler — Wed, 21 Mar 2007 09:41:00 -0500

Last week, I attended the Aspect-Oriented Software Development 2007 Conference in Vancouver, BC, where I gave a tutorial on aspect-oriented design and a paper in the Industry Track, also about design.

AOSD and this particular conference are still mostly academic projects with some notable industry traction, especially in the Java world. It is also a technology that needs to break through to the next level of innovation and applicability, in my opinion.

The Industry Track had a number of interesting papers, including, for example, a paper that describes how aspects are used in the innovative Terracotta JVM clustering tool. Also, the last keynote by Adrian Colyer of Interface 21 recounted his personal involvement in the AspectJ and Spring communities, as well as the impact that aspects are having at major Interface 21 clients. It’s worth noting that many of the important Java middleware systems, e.g., Spring, JBoss, and Weblogic have embraced aspects to one degree or another.

AOSD tools like AspectJ and Spring AOP (AO Programming) solve obvious “cross-cutting concerns” (CCCs) like object-relational mapping, transactions, security, etc. in Java. However, I feel that AOSD needs some breakthrough innovations to move beyond its current role as a useful niche technology to a more central role in the software development process. I get the impression that industry interest in AOP has reached a plateau.

The academic community is doing some interesting work on fundamentals of AOSD theory (type theory, modeling, categorizing types of aspects, etc.), on “early aspects” (e.g., cross-cutting requirements), and on tooling. There is also a lot of minor iterating around the edges, but that’s the nature of academic research (speaking as one who has been there…). Most of the research work is a long ways from practical applicability.

However, I’m seeing too much emphasis on extending the work of AspectJ-like approaches in statically-typed languages, rather than innovating in new areas like new applications of aspects and the nature of AOSD in dynamic languages.

My recent work with Ruby has made me think about these two topics lately. I’ll blog about these topics later, but for now, I’ll just say that I anticipate a fruitful growth area for AOSD will be to facilitate the implementation of powerful DSLs (Domain Specific Languages), a popular topic in the Ruby community.

Here are some other observations from the conference.

All other engineering disciplines recognize cross-cutting concerns

Gregor Kiczales (the father of AspectJ and one of the fathers of AOSD) made this remark in a panel discussion on “early aspects”. He cited the examples of electrical engineers considering systemic issues in circuit design, like capacitance, current leakage, etc. and mechanical engineers who evaluate the stresses and strains of the entire structure they are designing (buildings, brake assemblies in cars, etc.).

Gregor also remarked that CCCs that are evident in the requirements may disappear in the implementation and vice-versa.

Gerald Sussman keynote

In the first keynote, Gerald Sussman argued that robust systems are adaptable for uses that were not anticipated by their designers. These systems often have multiple organizational ideas and a “component” structure that promotes “combinatorial behavior”.

He doesn’t like formalisms such as Dykstra’s A Discipline of Programming. Provable correctness and rigor aren’t very compatible with rich programs. Sussman prefers a more “exploratory” model, very much analogous to Test Driven Development (TDD), and what he calls Paranoid Programming, which he defined as “I won’t be at fault if it fails.”

More on AOSD

I will blog further about aspect-oriented design and aspects in dynamic languages.

Liskov Substitution Principle and the Ruby Core Libraries

Dean Wampler — Sat, 17 Feb 2007 14:20:00 -0600

There is a spirited discussion happening now on the ruby-talk list called >Oppinions on RCR for dup on immutable classes (sic).

In the core Ruby classes, the Kernel module, which is the root of everything, even Object, defines a method called dup, for duplicating objects. (There is also a clone method with slightly different behavior that I won’t discuss here.)

The problem is that some derived core classes throw an exception when dup is called.

Specifically, as the ruby-talk discussion title says, it’s the immutable classes (NilClass, FalseClass, TrueClass, Fixnum, and Symbol) that do this. Consider, for example, the following irb session:

irb 1:0> 5.respond_to? :dup
=> true
irb 2:0> 5.dup
TypeError: can't dup Fixnum
        from (irb):1:in `dup'
        from (irb):1
irb 3:0>

If you don’t know Ruby, the first line asks the Fixnum object 5 if it responds to the method dup (with the name expressed as a symbol, hence the). The answer is true, becuase this method is defined by the module Kernel, which is included by the top-level class Object, an ancestor of Fixnum.

However, when you actually call dup on 5, it raises TypeError, as shown.

So, this looks like a classic Liskov Substitution Principle violation. The term for this code smell is Refused Bequest (e.g., see here) and it’s typically fixed with the refactoring Replace Inheritance with Delegation.

The email thread is about a proposal to change the library in one of several possible ways. One possibility is to remove dup from the immutable classes. This would eliminate the unexpected behavior in the example above, since 5.respond_to?(:dup) would return false, but it would still be an LSP violation, specifically it would still have the Refused Bequest smell.

One scenario where the current behavior causes problems is doing a deep copy of an arbitrary object graph. For immutable objects, you would normally just want dup to return the same object. It’s immutable, right? Well, not exactly, since you can re-open classes and even objects to add and remove methods in Ruby (there are some limitations for the immutables…). So, if you thought you actually duplicated something and started messing with its methods, you would be surprised to find the original was “also” modified.

So, how serious is this LSP issue (one of several)? When I pointed out the problem in the discussion, one respondent, Robert Dober, said the following (edited slightly):

I would say that LSP does not apply here simply because in Ruby we do not have that kind of contract. In order to apply LSP we need to say at a point we have an object of class Base, for example. (let the gods forgive me that I use Java)

void aMethod(final Base b){
   ....
}

and we expect this to work whenever we call aMethod with an object that is a Base. Anyway the compiler would not really allow otherwise.

SubClass sc;  // subclassing Base od course
aMethod( sc ); // this is expected to work (from the type POV).

Such things just do not exist in Ruby, I believe that Ruby has explained something to me:

OO Languages are Class oriented languages

Dynamic Languages are Object oriented languages.

Replace Class with Type and you see what I mean.

This is all very much IMHO of course but I feel that the Ruby community has made me evolve a lot away from “Class oriented”.

He’s wrong that the compiler protects you in Java; you can still throw exceptions, etc. The JDK Collection classes have Refused Bequests. Besides that, however, he makes some interesting points.

As a long-time Java programmer, I’m instinctively uncomfortable with LSP violations. Yet, the Ruby API is very nice to work with, so maybe a little LSP violation isn’t so bad?

As Robert says, we approach our designs differently in dynamic vs. static languages. In Ruby, you almost never use the is_a? and kind_of? methods to check for type. Instead, you follow the duck typing philosophy (“If it acts like a duck, it must be a duck”); you rely on respond_to? to decide if an object does what you want.

In the case of dup for the immutable classes, I would prefer that they not implement the method, rather than throw an exception. However, that would still violate LSP.

So, can we still satisfy LSP and also have rich base classes and modules?

There are many examples of traits that one object might or should support, but not another. (Those of you Java programmers might ask yourself why all objects support toString, for example. Why not also toXML...?)

Coming from an AOP background, I would rather see an architecture where dup is added only to those classes and modules that can support it. It shouldn’t be part of the standard “signature” of Kernel, but it should be present when code actually needs it.

In fact, Ruby makes this sort of AOP easy to implement. Maybe Kernel, Module, and Object should be refactored into smaller pieces and programmers should declaratively mixin the traits they need. Imagine something like the following:

irb 1:0> my_obj.respond_to? :dup
=> false
irb 2:0> include 'DupableTrait'  
irb 2:0> my_obj.respond_to? :dup
=> true
irb 4:0> def dup_if_possible items
irb 5:1>  items.map {|item| item.respond_to?(:dup) ? item.dup : item}
irb 6:1> end
...

In other words, Kernel no longer “exposes the dup abstraction”, by default, but the DupableTrait module “magically” adds dup to all the classes that can support it. This way, we preserve LSP, streamline the core classes and modules (SRP and ISP anyone?), yet we have the flexibility we need, on demand.

Phantom (Menace) Dependencies

Dean Wampler — Fri, 26 Jan 2007 18:07:00 -0600

We’ve been working on new course materials lately. The other day we were discussing the bad dependencies in code that can occur when the Interface Segregation Principle (ISP) is violated.

We started calling these dependencies phantom dependencies. Here’s why…

In the simplest example of an ISP problem, several client components depend on a “server” component, but each uses completely independent facilities provided by that component. There are no direct or indirect connections between clients. So, none of the clients knows anything about the other clients, nor cares to, yet each client has an implicit dependency on the other clients.

Why? Because if a change in one client forces a change in the server component, the other clients are affected. At the very least, a rebuild may be required.

The solution is to hide the server component behind segregated interfaces, one tailored for each client, and have the server component implement those interfaces. Even better, if the features really are independent, then segregate the server component, too!

Anyway, we sometimes use the term back-channel dependency for this situation. However, while thinking about it, I remembered an article that I had read a few days before on the phantom pain phenomenon that amputees sometimes experience, where they feel pain in limbs that are no longer there. The ISP case is analogous; there is no traversable (“real”) connections from one client component to another, yet each “feels” the presence of the others. Hence, the term phantom dependency.

Protecting Developers from Powerful Languages

Dean Wampler — Mon, 15 Jan 2007 21:57:00 -0600

Microsoft’s forthcoming C# version 3 has some innovative features, as described in this blog. I give the C# team credit for pushing the boundaries of C#, in part because they have forced the Java community to follow suit. ;)

A common tension in many development shops is how far to trust the developers with languages and tools that are perceived to be “advanced”. It’s tempting to limit developers to “safe” languages and maybe not all the features of those languages. This can be misguided.

Java is usually considered safe, but Java Generics are suspect. Strong typing is safe, but dynamic typing isn’t controlled enough. Closures and continuations sound too advanced and technical to be trusted in the hands of “our team”.

To be fair, larger organizations have more at stake and caution is prudent. Regrettably, it is also true that many people in our profession are … hmm … not that well qualified.

However, I find that I’m far more productive and less likely to make mistakes using Ruby iterators with closures than writing more verbose and inelegant Java.

I used to be a strong believer in static typing, but it has become a distraction, as I have to worry more about the types of method parameters and return values, rather than just worrying about the values themselves. I realized that, on average in a typical section of code, the actual type of a variable is unimportant. The variable is just a “handle” being passed around. The name is always important, as it is a form of documentation. There are places where the type is important, of course, when the variable is read or written in some way.

Finally, static typing offers less security than at first appears. At best, it only confirms that variables of particular types are used consistently. Your unit tests also do this. However, static typing can’t confirm that the usage of the API is correct. This is analogous to testing the syntax but not the semantics of the program. In fact, only unit tests (or alternatives, like rspec ) are effective at testing both.

So, it’s prudent to be reticent about newer languages and features, but make sure the decisions you make about them are backed up by careful evaluation and don’t forget to train your team appropriately!