Object Mentor Blog: Category Design Principles

Baubles in Orbit

Uncle Bob — Tue, 19 Aug 2008 01:29:32 +0000

I have put together a nice little demonstration of the Bauble concept. You may recall that I first wrote about it here. Baubles are a simple component scheme for Ruby, good for when you want a component, but don’t need something as heavy as a gem.

orbit.zip contains all the files for this demonstration. I suggest you download and unpack it.

First you need to install the Bauble gem. Don’t worry, it won’t hurt anything. Just say gem install orbit/bauble/Bauble-0.1.gem (You’ll probably have to do it with sudo.) That should install Bauble. From now on you only need to say require 'bauble' in your ruby scripts that make use of it.

Now you should be able to run the orbital simulator. Just type:


cd orbit/MultipleBodyOrbit/lib
jruby multiple_body_orbit.rb

A swing window should pop up and you should be able to watch an orbital simulation. Every run shows a different random scenario, so you can kill a lot of time by watching worlds in collision.

The thing to note, if you are a ruby programmer, is the use of the term Bauble::use(-some_directory-). If you look in the multiple_body_orbit.rb file you’ll see I use two Baubles, the Physics bauble does the raw calculation for all the gravity, forces, collisions, etc. The cellular_automaton bauble provides a very simple Swing framework for drawing dots on a screen. (Yes, this is jruby).

If you look in either of the two Baubles, you’ll see that the require statements within them do not know (or care) about the directory they live in. There is none of that horrible __FILE__ nonsense that pollutes so many ruby scripts. This is because the Bauble::use function puts the directory path in the LOAD_PATH so that subsquent require statements can simply eliminate the directory spec.

Take a look at the Bauble source code. It’s no great shakes.

Also take a look at the two baubles. They show a pretty nice way to decouple business rules from gui. You might recognize the MVP pattern. The multiple_body_orbit.rb file contains the presenter. Clearly the Physics module is the model. And the cellular_automaton module is the view. (There is no controller, because there is no input.)

The Seductions of Scala, Part III - Concurrent Programming

Dean Wampler — Thu, 14 Aug 2008 16:00:00 +0000

This is my third and last blog entry on The Seductions of Scala, where we’ll look at concurrency using Actors and draw some final conclusions.

Writing Robust, Concurrent Programs with Scala

The most commonly used model of concurrency in imperative languages (and databases) uses shared, mutable state with access synchronization. (Recall that synchronization isn’t necessary for reading immutable objects.)

However, it’s widely known that this kind of concurrency programming is very difficult to do properly and few programmers are skilled enough to write such programs.

Because pure functional languages have no side effects and no shared, mutable state, there is nothing to synchronize. This is the main reason for the resurgent interest in function programming recently, as a potential solution to the so-called multicore problem.

Instead, most functional languages, in particular, Erlang and Scala, use the Actor model of concurrency, where autonomous “objects” run in separate processes or threads and they pass messages back and forth to communicate. The simplicity of the Actor model makes it far easier to create robust programs. Erlang processes are so lightweight that it is common for server-side applications to have thousands of communicating processes.

Actors in Scala

Let’s finish our survey of Scala with an example using Scala’s Actors library.

Here’s a simple Actor that just counts to 10, printing each number, one per second.

    
import scala.actors._
object CountingActor extends Actor { 
    def act() { 
        for (i <- 1 to 10) { 
            println("Number: "+i)
            Thread.sleep(1000) 
        } 
    } 
} 

CountingActor.start()

The last line starts the actor, which implicitly invokes the act method. This actor does not respond to any messages from other actors.

Here is an actor that responds to messages, echoing the message it receives.

    
import scala.actors.Actor._ 
val echoActor = actor {
    while (true) {
        receive {
            case msg => println("received: "+msg)
        }
    }
}
echoActor ! "hello" 
echoActor ! "world!"

In this case, we do the equivalent of a Java “static import” of the methods on Actor, e.g., actor. Also, we don’t actually need a special class, we can just create an object with the desired behavior. This object has an infinite loop that effectively blocks while waiting for an incoming message. The receive method gets a block that is a match statement, which matches on anything received and prints it out.

Messages are sent using the target_actor ! message syntax.

As a final example, let’s do something non-trivial; a contrived network node monitor.

    
import scala.actors._
import scala.actors.Actor._
import java.net.InetAddress 
import java.io.IOException

case class NodeStatusRequest(address: InetAddress, respondTo: Actor) 

sealed abstract class NodeStatus
case class Available(address: InetAddress) extends NodeStatus
case class Unresponsive(address: InetAddress, reason: Option[String]) extends NodeStatus

object NetworkMonitor extends Actor {
    def act() {
        loop {
            react {  // Like receive, but uses thread polling for efficiency.
                case NodeStatusRequest(address, actor) => 
                    actor ! checkNodeStatus(address)
                case "EXIT" => exit()
            }
        }
    }
    val timeoutInMillis = 1000;
    def checkNodeStatus(address: InetAddress) = {
        try {
            if (address.isReachable(timeoutInMillis)) 
                Available(address)
            else
                Unresponsive(address, None)
        } catch {
            case ex: IOException => 
                Unresponsive(address, Some("IOException thrown: "+ex.getMessage()))
        }
    }
}

// Try it out:

val allOnes = Array(1, 1, 1, 1).map(_.toByte)
NetworkMonitor.start()
NetworkMonitor ! NodeStatusRequest(InetAddress.getByName("www.scala-lang.org"), self)
NetworkMonitor ! NodeStatusRequest(InetAddress.getByAddress("localhost", allOnes), self)
NetworkMonitor ! NodeStatusRequest(InetAddress.getByName("www.objectmentor.com"), self)
NetworkMonitor ! "EXIT" 
self ! "No one expects the Spanish Inquisition!!" 

def handleNodeStatusResponse(response: NodeStatus) = response match {
    // Sealed classes help here
    case Available(address) => 
        println("Node "+address+" is alive.")
    case Unresponsive(address, None) => 
        println("Node "+address+" is unavailable. Reason: <unknown>")
    case Unresponsive(address, Some(reason)) => 
        println("Node "+address+" is unavailable. Reason: "+reason)
}

for (i <- 1 to 4) self.receive {   // Sealed classes don't help here
    case (response: NodeStatus) => handleNodeStatusResponse(response)
    case unexpected => println("Unexpected response: "+unexpected)
}

We begin by importing the Actor classes, the methods on Actor, like actor, and a few Java classes we need.

Next we define a sealed abstract base class. The sealed keyword tells the compiler that the only subclasses will be defined in this file. This is useful for the case statements that use them. The compiler will know that it doesn’t have to worry about potential cases that aren’t covered, if new NodeStatus subclasses are created. Otherwise, we would have to add a default case clause (e.g., case _ => ...) to prevent warnings (and possible errors!) about not matching an input. Sealed class hierarchies are a useful feature for robustness (but watch for potential Open/Closed Principle violations!).

The sealed class hierarchy encapsulates all the possible node status values (somewhat contrived for the example). The node is either Available or Unresponsive. If Unresponsive, an optional reason message is returned.

Note that we only get the benefit of sealed classes here because we match on them in the handleNodeStatusResponse message, which requires a response argument of type NodeStatus. In contrast, the receive method effectively takes an Any argument, so sealed classes don’t help on the line with the comment “Sealed classes don’t help here”. In that case, we really need a default, the case unexpected => ... clause. (I added the message self ! "No one expects the Spanish Inquisition!!" to test this default handler.)

In the first draft of this blog post, I didn’t know these details about sealed classes. I used a simpler implementation that couldn’t benefit from sealed classes. Thanks to the first commenter, LaLit Pant, who corrected my mistake!

The NetworkMonitor loops, waiting for a NodeStatusRequest or the special string “EXIT”, which tells it to quit. Note that the actor sending the request passes itself, so the monitor can reply to it.

The checkNodeStatus attempts to contact the node, with a 1 second timeout. It returns an appropriate NodeStatus.

Then we try it out with three addresses. Note that we pass self as the requesting actor. This is an Actor wrapping the current thread, imported from Actor. It is analogous to Java’s Thread.currentThread().

Curiously enough, when I run this code, I get the following results.

    
Unexpected response: No one expects the Spanish Inquisition!!
Node www.scala-lang.org/128.178.154.102 is unavailable. Reason: <unknown>
Node localhost/1.1.1.1 is unavailable. Reason: <unknown>
Node www.objectmentor.com/206.191.6.12 is alive.

The message about the Spanish Inquisition was sent last, but processed first, probably because self sent it to itself.

I’m not sure why www.scala-lang.org couldn’t be reached. A longer timeout didn’t help. According to the Javadocs for InetAddress.isReachable), it uses ICMP ECHO REQUESTs if the privilege can be obtained, otherwise it tries to establish a TCP connection on port 7 (Echo) of the destination host. Perhaps neither is supported on the scala-lang.org site.

Conclusions

Here are some concluding observations about Scala vis-à-vis Java and other options.

A Better Java

Ignoring the functional programming aspects for a moment, I think Scala improves on Java in a number of very useful ways, including:

A more succinct syntax. There’s far less boilerplate, like for fields and their accessors. Type inference and optional semicolons, curly braces, etc. also reduce “noise”.
A true mixin model. The addition of traits solves the problem of not having a good DRY way to mix in additional functionality declared by Java interfaces.
More flexible method names and invocation syntax. Java took away operator overloading; Scala gives it back, as well as other benefits of using non-alphanumeric characters in method names. (Ruby programmers enjoy writing list.empty?, for example.)
Tuples. A personal favorite, I’ve always wanted the ability to return multiple values from a method, without having to create an ad hoc class to hold the values.
Better separation of mutable vs. immutable objects. While Java provides some ability to make objects final, Scala makes the distinction between mutability and immutability more explicit and encourages the latter as a more robust programming style.
First-class functions and closures. Okay, these last two points are really about FP, but they sure help in OO code, too!
Better mechanisms for avoiding null’s. The Option type makes code more robust than allowing null values.
Interoperability with Java libraries. Scala compiles to byte code so adding Scala code to existing Java applications is about as seamless as possible.

So, even if you don’t believe in FP, you will gain a lot just by using Scala as a better Java.

Functional Programming

But, you shouldn’t ignore the benefits of FP!

Better robustness. Not only for concurrent programs, but using immutable objects (a.k.a. value objects) reduces the potential for bugs.
A workable concurrency model. I use the term workable because so few developers can write robust concurrent code using the synchronization on shared state model. Even for those of you who can, why bother when Actors are so much easier??
Reduced code complexity. Functional code tends to be very succinct. I can’t overestimate the importance of rooting out all accidental complexity in your code base. Excess complexity is one of the most pervasive detriments to productivity and morale that I see in my clients’ code bases!
First-class functions and closures. Composition and succinct code are much easier with first-class functions.
Pattern matching. FP-style pattern matching makes “routing” of messages and delegation much easier.

Of course, you can mimic some of these features in pure Java and I encourage you to do so if you aren’t using Scala.

Static vs. Dynamic Typing

The debate on the relative merits of static vs. dynamic typing is outside our scope, but I will make a few personal observations.

I’ve been a dedicated Rubyist for a while. It is hard to deny the way that dynamic typing simplifies code and as I said in the previous section, I take code complexity very seriously.

Scala’s type system and type inference go a long way towards providing the benefits of static typing with the cleaner syntax of dynamic typing, but Scala doesn’t eliminate the extra complexity of static typing.

Recall my Observer example from the first blog post, where I used traits to implement it.

    
trait Observer[S] {
    def receiveUpdate(subject: S);
}

trait Subject[S] { 
    this: S =>
    private var observers: List[Observer[S]] = Nil
    def addObserver(observer: Observer[S]) = observers = observer :: observers

    def notifyObservers() = observers.foreach(_.receiveUpdate(this))
}

In Ruby, we might implement it this way.

    
module Subject 
    def add_observer(observer) 
      @observers ||= []
      @observers << observer  # append, rather than replace with new array
  end

    def notify_observers
      @observers.each {|o| o.receive_update(self)} if @observers
  end
end

There is no need for an Observer module. As long as every observer responds to the receive_update “message”, we’re fine.

I commented the line where I append to the existing @observers array, rather than build a new one, which would be the FP and Scala way. Appending to the existing array would be more typical of Ruby code, but this implementation is not as thread safe as an FP-style approach.

The trailing if expression in notify_observers means that nothing is done if @observers is still nil, i.e., it was never initialized in add_observer.

So, which is better? The amount of code is not that different, but it took me significantly longer to write the Scala version. In part, this was due to my novice chops, but the reason it took me so long was because I had to solve a design issue resulting from the static typing. I had to learn about the typed self construct used in the first line of the Subject trait. This was the only way to allow the Observer.receiveUpdate method accept to an argument of type S, rather than of type Subject[S]. It was worth it to me to achieve the “cleaner” API.

Okay, perhaps I’ll know this next time and spend about the same amount of time implementing a Ruby vs. Scala version of something. However, I think it’s notable that sometimes static typing can get in the way of your intentions and goal of achieving clarity. (At other times, the types add useful documentation.) I know this isn’t the only valid argument you can make, one way or the other, but it’s one reason that dynamic languages are so appealing.

Poly-paradigm Languages vs. Mixing Several Languages

So, you’re convinced that you should use FP sometimes and OOP sometimes. Should you pick a poly-paradigm language, like Scala? Or, should you combine several languages, each of which implements one paradigm?

A potential downside of Scala is that supporting different modularity paradigms, like OOP and FP, increases the complexity in the language. I think Odersky and company have done a superb job combining FP and OOP in Scala, but if you compare Scala FP code to Haskell or Erlang FP code, the latter tend to be more succinct and often easier to understand (once you learn the syntax).

Indeed, Scala will not be easy for developers to master. It will be a powerful tool for professionals. As a consultant, I work with developers with a range of skills. I would not expect some of them to prosper with Scala. Should that rule out the language? NO. Rather it would be better to “liberate” the better developers with a more powerful tool.

So, if your application needs OOP and FP concepts interspersed, consider Scala. If your application needs discrete services, some of which are predominantly OOP and others of which are predominantly FP, then consider Scala or Java for the OOP parts and Erlang or another FP language for the FP parts.

Also, Erlang’s Actor model is more mature than Scala’s, so Erlang might have an edge for a highly-concurrent server application.

Of course, you should do your own analysis…

Final Thoughts

Java the language has had a great ride. It was a godsend to us beleaguered C++ programmers in the mid ‘90’s. However, compared to Scala, Java now feels obsolete. The JVM is another story. It is arguably the best VM available.

I hope Scala replaces Java as the main JVM language for projects that prefer statically-typed languages. Fans of dynamically-typed languages might prefer JRuby, Groovy, or Jython. It’s hard to argue with all the OOP and FP goodness that Scala provides. You will learn a lot about good language and application design by learning Scala. It will certainly be a prominent tool in my toolkit from now on.

The Seductions of Scala, Part II - Functional Programming

Dean Wampler — Tue, 05 Aug 2008 20:32:00 +0000

A Functional Programming Language for the JVM

In my last blog post, I discussed Scala’s support for OOP and general improvements compared to Java. In this post, which I’m posting from Agile 2008, I discuss Scala’s support for functional programming (FP) and why it should be of interest to OO developers.

A Brief Overview of Functional Programming

You might ask, don’t most programming languages have functions? FP uses the term in the mathematical sense of the word. I hate to bring up bad memories, but you might recall from your school days that when you solved a function like

    
y = sin(x)

for y, given a value of x, you could input the same value of x an arbitrary number of times and you would get the same value of y. This means that sin(x) has no side effects. In other words, unlike our imperative OO or procedural code, no global or object state gets changed. All the work that a mathematical function does has to be returned in the result.

Similarly, the idea of a variable is a little different than what we’re used to in imperative code. While the value of y will vary with the value of x, once you have fixed x, you have also fixed y. The implication for FP is that “variables” are immutable; once assigned, they cannot be changed. I’ll call such immutable variables value objects.

Now, it would actually be hard for a “pure” FP language to have no side effects, ever. I/O would be rather difficult, for example, since the state of the input or output stream changes with each operation. So, in practice, all “pure” FP languages provide some mechanisms for breaking the rules in a controlled way.

Functions are first-class objects in FP. You can create named or anonymous functions (e.g., closures or blocks), assign them to variables, pass them as arguments to other functions, etc. Java doesn’t support this. You have to create objects that wrap the methods you want to invoke.

Functional programs tend to be much more declarative in nature than imperative programs. This is perhaps more obvious in pure FP languages, like Erlang and Haskell, than it is in Scala.

For example, the definition of Fibonacci numbers is the following.

    
F(n) = F(n-1) + F(n-2) where F(1)=1 and F(2)=1

An here is a complete implementation in Haskell.

    
module Main where 
-- Function f returns the n'th Fibonacci number. 
-- It uses binary recursion. 
f n | n <= 2 = 1 
    | n >  2 = f (n-1) + f (n-2)

Without understanding the intricacies of Haskell syntax, you can see that the code closely matches the “specification” above it. The f n | ... syntax defines the function f taking an argument n and the two cases of n values are shown on separate lines, where one case is for n <= 2 and the other case if for n > 2.

The code uses the recursive relationship between different values of the function and the special-case values when n = 1 and n = 2. The Haskell runtime does the rest of the work.

It’s interesting that most domain-specific languages are also declarative in nature. Think of how JMock, EasyMock or Rails’ ActiveRecord code look. The code is more succinct and it lets the “system” do most of the heavy lifting.

Functional Programming’s Benefits for You

Value Objects and Side-Effect Free Functions

It’s the immutable variables and side-effect free functions that help solve the multicore problem. Synchronized access to shared state is not required if there is no state to manage. This makes robust concurrent programs far easier to write.

I’ll discuss concurrency in Scala in my third post. For now, let’s discuss other ways that FP in Scala helps to improve code, concurrent or not.

Value objects are beneficial because you can pass one around without worrying that someone will change it in a way that breaks other users of the object. Value objects aren’t unique to FP, of course. They have been promoted in Domain Driven Design (DDD), for example.

Similarly, side-effect free functions are safer to use. There is less risk that a caller will change some state inappropriately. The caller doesn’t have to worry as much about calling a function. There are fewer surprises and everything of “consequence” that the function does is returned to the caller. It’s easier to keep to the Single Responsibility Principle when writing side-effect free functions.

Of course, you can write side-effect free methods and immutable variables in Java code, but it’s mostly a matter of discipline; the language doesn’t give you any enforcement mechanisms.

Scala gives you a helpful enforcement mechanism; the ability to declare variables as val’s (i.e., “values”) vs. var’s (i.e., “variables”, um… back to the imperative programming sense of the word…). In fact, val is the default, where neither is required by the language. Also, the Scala library contains both immutable and mutable collections and it “encourages” you to use the immutable collections.

However, because Scala combines both OOP and FP, it doesn’t force FP purity. The upside is that you get to use the approach that best fits the problem you’re trying to solve. It’s interesting that some of the Scala library classes expose FP-style interfaces, immutability and side-effect free functions, while using more traditional imperative code to implement them!

Closures and First-Class Functions

True to its functional side, Scala gives you true closures and first-class functions. If you’re a Groovy or Ruby programmer, you’re used to the following kind of code.

    
class ExpensiveResource {
    def open(worker: () => Unit) = {
        try {
            println("Doing expensive initialization")
            worker()
        } finally {
            close()
        }
    }
    def close() = {
        println("Doing expensive cleanup")
    }
}
// Example use:
try {
    (new ExpensiveResource()) open { () =>        // 1
        println("Using Resource")                 // 2
        throw new Exception("Thrown exception")   // 3
    }                                             // 4
} catch {
    case ex: Throwable => println("Exception caught: "+ex)
}

Running this code will yield:

    
Doing expensive initialization
Using Resource
Doing expensive cleanup
Exception caught: java.lang.Exception: Thrown exception

The ExpensiveResource.open method invokes the user-specified worker function. The syntax worker: () => Unit defines the worker parameter as a function that takes no arguments and returns nothing (recall that Unit is the equivalent of void).

ExpensiveResource.open handles the details of initializing the resource, invoking the worker, and doing the necessary cleanup.

The example marked with the comment // 1 creates a new ExpensiveResource, then calls open, passing it an anonymous function, called a function literal in Scala terminology. The function literal is of the form (arg_list_) => function body or () => println(...) ..., in our case.

A special syntax trick is used on this line; if a method takes one argument, you can change expressions of the form object.method(arg) to object method {arg}. This syntax is supported to allow user-defined methods to read like control structures (think for statements – see the next section). If you’re familiar with Ruby, the four commented lines read a lot like Ruby syntax for passing blocks to methods.

Idioms like this are very important. A library writer can encapsulate all complex, error-prone logic and allow the user to specify only the unique work required in a given situation. For example, How many times have you written code that opened an I/O stream or a database connection, used it, then cleaned up. How many times did you get the idiom wrong, especially the proper cleanup when an exception is thrown? First-class functions allow writers of I/O, database and other resource libraries to do the correct implementation once, eliminating user error and duplication. Here’s a rhetorical question I always ask myself:

How can I make it impossible for the user of this API to fail?

Iterations

Iteration through collections, Lists in particular, is even more common in FP than in imperative languages. Hence, iteration is highly evolved. Consider this example:

    
object RequireWordsStartingWithPrefix {
    def main(args: Array[String]) = {
        val prefix = args(0)
        for {
            i <- 1 to (args.length - 1)   // no semicolon
            if args(i).startsWith(prefix)
        } println("args("+i+"): "+args(i))
    }
}

Compiling this code with scalac and then running it on the command line with the command

    
scala RequireWordsStartingWithPrefix xx xy1 xx1 yy1 xx2 xy2

produces the result

    
args(2): xx1
args(5): xx2

The for loop assigns a loop variable i with each argument, but only if the if statement is true. Instead of curly braces, the for loop argument list could also be parenthesized, but then each line as shown would have to be separated by a semi-colon, like we’re used to seeing with Java for loops.

We can have an arbitrary number of assignments and conditionals. In fact, it’s quite common to filter lists:

    
object RequireWordsStartingWithPrefix2 {
    def main(args: Array[String]) = {
        val prefix = args(0)
        args.slice(1, args.length)
            .filter((arg: String) => arg.startsWith(prefix))
            .foreach((arg: String) => println("arg: "+arg))
    }
}

This version yields the same result. In this case, the args array is sliced (loping off the search prefix), the resulting array is filtered using a function literal and the filtered array is iterated over to print out the matching arguments, again using a function literal. This version of the algorithm should look familiar to Ruby programmers.

Rolling Your Own Function Objects

Scala still has to support the constraints of the JVM. As a comment to the first blog post said, the Scala compiler wraps closures and “bare” functions in Function objects. You can also make other objects behave like functions. If your object implements the apply method, that method will be invoked if you put parentheses with an matching argument list on the object, as in the following example.

    
class HelloFunction {
    def apply() = "hello" 
    def apply(name: String) = "hello "+name
}
val hello = new HelloFunction
println(hello())        // => "hello" 
println(hello("Dean"))  // => "hello Dean"

Option, None, Some…

Null pointer exceptions suck. You can still get them in Scala code, because Scala runs on the JVM and interoperates with Java libraries, but Scala offers a better way.

Typically, a reference might be null when there is nothing appropriate to assign to it. Following the conventions in some FP languages, Scala has an Option type with two subtypes, Some, which wraps a value, and None, which is used instead of null. The following example, which also demonstrates Scala’s Map support, shows these types in action.

    
val hotLangs = Map(
    "Scala" -> "Rocks", 
    "Haskell" -> "Ethereal", 
    "Java" -> null)
println(hotLangs.get("Scala"))          // => Some(Rocks)
println(hotLangs.get("Java"))           // => Some(null)
println(hotLangs.get("C++"))            // => None

Note that Map stores values in Options objects, as shown by the println statements.

By the way, those -> aren’t special operators; they’re methods. Like ::, valid method names aren’t limited to alphanumerics, _, and $.

Pattern Matching

The last FP feature I’ll discuss in this post is pattern matching, which is exploited more fully in FP languages than in imperative languages.

Using our previous definition of hotLangs, here’s how you might use matching.

    
def show(key: String) = {
    val value: Option[String] = hotLangs.get(key)
    value match {
        case Some(x) => x
        case None => "No hotness found" 
    }
}
println(show("Scala"))  // => "Rocks" 
println(show("Java"))   // => "null" 
println(show("C++"))    // => "No hotness found"

The first case statement, case Some(x) => x, says “if the value I’m matching against is a Some that could be constructed with the Some[+String](x: A) constructor, then return the x, the thing the Some contains.” Okay, there’s a lot going on here, so more background information is in order.

In Scala, like Ruby and other languages, the last value computed in a function is returned by it. Also, almost everything returns a value, including match statements, so when the Some(x) => x case is chosen, x is returned by the match and hence by the function.

Some is a generic class and the show function returns a String, so the match is to Some[+String]. The + in the +String expression is analogous to Java’s extends, i.e., <? extends String>. Capiche?

Idioms like case Some(x) => x are called extractors in Scala and are used a lot in Scala, as well as in FP, in general. Here’s another example using Lists and our friend ::, the “cons” operator.

    
def countScalas(list: List[String]): Int = {
    list match {
        case "Scala" :: tail => countScalas(tail) + 1
        case _ :: tail       => countScalas(tail)
        case Nil             => 0
    }
}
val langs = List("Scala", "Java", "C++", "Scala", "Python", "Ruby")
val count = countScalas(langs)
println(count)    // => 2

We’re counting the number of occurrences of “Scala” in a list of strings, using matching and recursion and no explicit iteration. An expression of the form head :: tail applied to a list returns the first element set as the head variable and the rest of the list set as the tail variable. In our case, the first case statement looks for the particular case where the head equals Scala. The second case matches all lists, except for the empty list (Nil). Since matches are eager, the first case will always pick out the List("Scala", ...) case first. Note that in the second case, we don’t actually care about the value, so we use the placeholder _. Both the first and second case’s call countScalas recursively.

Pattern matching like this is powerful, yet succinct and elegant. We’ll see more examples of matching in the next blog post on concurrency using message passing.

Recap of Scala’s Functional Programming

I’ve just touched the tip of the iceberg concerning functional programming (and I hope I got all the details right!). Hopefully, you can begin to see why we’ve overlooked FP for too long!

In my last post, I’ll wrap up with a look at Scala’s approach to concurrency, the Actor model of message passing.

The Seductions of Scala, Part I

Dean Wampler — Sun, 03 Aug 2008 15:30:00 +0000

Because of all the recent hoo-ha about functional programming (e.g., as a “cure” for the multicore problem), I decided to cast aside my dysfunctional ways and learn one of the FP languages. The question was, which one?

My distinguished colleague, Michael Feathers, has been on a Haskell binge of late. Haskell is a pure functional language and is probably most interesting as the “flagship language” for academic exploration, rather than production use. (That was not meant as flame bait…) It’s hard to underestimate the influence Haskell has had on language design, including Java generics, .NET LINQ and F#, etc.

However, I decided to learn Scala first, because it is a JVM language that combines object-oriented and functional programming in one language. At ~13 years of age, Java is a bit dated. Scala has the potential of replacing Java as the principle language of the JVM, an extraordinary piece of engineering that is arguably now more valuable than the language itself. (Note: there is also a .NET version of Scala under development.)

Here are some of my observations, divided over three blog posts.

First, a few disclaimers. I am a Scala novice, so any flaws in my analysis reflect on me, not Scala! Also, this is by no means an exhaustive analysis of the pros and cons of Scala vs. other options. Start with the Scala website for more complete information.

A Better OOP Language

Scala works seamlessly with Java. You can invoke Java APIs, extend Java classes and implement Java interfaces. You can even invoke Scala code from Java, once you understand how certain “Scala-isms” are translated to Java constructs (javap is your friend). Scala syntax is more succinct and removes a lot of tedious boilerplate from Java code.

For example, the following Person class in Java:

    
class Person {
    private String firstName;
    private String lastName;
    private int    age;

    public Person(String firstName, String lastName, int age) {
        this.firstName = firstName;
        this.lastName  = lastName;
        this.age       = age;
    }

    public void setFirstName(String firstName) { this.firstName = firstName; }
    public void String getFirstName() { return this.firstName; }
    public void setLastName(String lastName) { this.lastName = lastName; }
    public void String getLastName() { return this.lastName; }
    public void setAge(int age) { this.age = age; }
    public void int getAge() { return this.age; }
}

can be written in Scala thusly:

    
class Person(var firstName: String, var lastName: String, var age: Int)

Yes, that’s it. The constructor is the argument list to the class, where each parameter is declared as a variable (var keyword). It automatically generates the equivalent of getter and setter methods, meaning they look like Ruby-style attribute accessors; the getter is foo instead of getFoo and the setter is foo = instead of setFoo. Actually, the setter function is really foo_=, but Scala lets you use the foo = sugar.

Lots of other well designed conventions allow the language to define almost everything as a method, yet support forms of syntactic sugar like the illusion of operator overloading, Ruby-like DSL’s, etc.

You also get fewer semicolons, no requirements tying package and class definitions to the file system structure, type inference, multi-valued returns (tuples), and a better type and generics model.

One of the biggest deficiencies of Java is the lack of a complete mixin model. Mixins are small, focused (think Single Responsibility Principle ...) bits of state and behavior that can be added to classes (or objects) to extend them as needed. In a language like C++, you can use multiple inheritance for mixins. Because Java only supports single inheritance and interfaces, which can’t have any state and behavior, implementing a mixin-based design has always required various hacks. Aspect-Oriented Programming is also one partial solution to this problem.

The most exciting OOP enhancement Scala brings is its support for Traits, a concept first described here. Traits support Mixins (and other design techniques) through composition rather than inheritance. You could think of traits as interfaces with implementations. They work a lot like Ruby modules.

Here is an example of the Observer Pattern written as traits, where they are used to monitor changes to a bank account balance. First, here are reusable Subject and Observer traits.

    
trait Observer[S] {
    def receiveUpdate(subject: S);
}

trait Subject[S] { 
    this: S =>
    private var observers: List[Observer[S]] = Nil
    def addObserver(observer: Observer[S]) = observers = observer :: observers

    def notifyObservers() = observers.foreach(_.receiveUpdate(this))
}

In Scala, generics are declared with square brackets, [...], rather than angled brackets, <...>. Method definitions begin with the def keyword. The Observer trait defines one abstract method, which is called by the Subject to notify the observer of changes. The Subject is passed to the Observer.

This trait looks exactly like a Java interface. In fact, that’s how traits are represented in Java byte code. If the trait has state and behavior, like Subject, the byte code representation involves additional elements.

The Subject trait is more complex. The strange line, this: S => , is called a self type declaration. It tells the compiler that whenever this is referenced in the trait, treat its type as S, rather than Subject[S]. Without this declaration, the call to receiveUpdate in the notifyObservers method would not compile, because it would attempt to pass a Subject[S] object, rather than a S object. The self type declaration solves this problem.

The next line creates a private list of observers, initialized to Nil, which is an empty list. Variable declarations are name: type. Why didn’t they follow Java conventions, i.e., type name? Because this syntax makes the code easier to parse when type inference is used, meaning where the explicit :type is omitted and inferred.

In fact, I’m using type inference for all the method declarations, because the compiler can figure out what each method returns, in my examples. In this case, they all return type Unit, the equivalent of Java’s void. (The name Unit is a common term in functional languages.)

The third line defines a method for adding a new observer to the list. Notice that concrete method definitions are of the form

    
def methodName(parameter: type, ...) = {
    method body
}

In this case, because there is only one line, I dispensed with the {...}. The equals sign before the body emphasizes the functional nature of scala, that all methods are objects, too. We’ll revisit this in a moment and in the next post.

The method body prepends the new observer object to the existing list. Actually, a new list is created. The :: operator, called “cons”, binds to the right. This “operator” is really a method call, which could actually be written like this, observers.::(observer).

Our final method in Subject is notifyObservers. It iterates through observers and invokes the block observer.receiveUpdate(this) on each observer. The _ evaluates to the current observer reference. For comparison, in Ruby, you would define this method like so:

    
def notifyObservers() 
    @observers.each { |o| o.receiveUpdate(self) }
end

Okay, let’s look at how you would actually use these traits. First, our “plain-old Scala object” (POSO) Account.

    
class Account(initialBalance: Double) {
    private var currentBalance = initialBalance
    def balance = currentBalance
    def deposit(amount: Double)  = currentBalance += amount
    def withdraw(amount: Double) = currentBalance -= amount
}

Hopefully, this is self explanatory, except for two things. First, recall that the whole class declaration is actually the constructor, which is why we have an initialBalance: Double parameter on Account. This looks strange to the Java-trained eye, but it actually works well and is another example of Scala’s economy. (You can define multiple constructors, but I won’t go into that here…).

Second, note that I omitted the parentheses when I defined the balance “getter” method. This supports the uniform access principle. Clients will simply call myAccount.balance, without parentheses and I could redefine balance to be a var or val and the client code would not have to change!

Next, a subclass that supports observation.

    
class ObservedAccount(initialBalance: Double) extends Account(initialBalance) with Subject[Account] {
    override def deposit(amount: Double) = {
        super.deposit(amount)
        notifyObservers()
    }
    override def withdraw(amount: Double) = {
        super.withdraw(amount)
        notifyObservers()
    }
}

The with keyword is how a trait is used, much the way that you implement an interface in Java, but now you don’t have to implement the interface’s methods. We’ve already done that.

Note that the expression, ObservedAccount(initialBalance: Double) extends Account(initialBalance), not only defines the (single) inheritance relationship, it also functions as the constructor’s call to super(initialBalance), so that Account is properly initialized.

Next, we have to override the deposit and withdraw methods, calling the parent methods and then invoking notifyObservers. Anytime you override a concrete method, scala requires the override keyword. This tells you unambiguously that you are overriding a method and the Scala compiler throws an error if you aren’t actually overriding a method, e.g., because of a typo. Hence, the keyword is much more reliable (and hence useful…) than Java’s @Override annotation.

Finally, here is an Observer that prints to stdout when the balance changes.

    
class AccountReporter extends Observer[Account] {
    def receiveUpdate(account: Account) =
        println("Observed balance change: "+account.balance)
}

Rather than use with, I just extend the Observer trait, because I don’t have another parent class.

Here’s some code to test what we’ve done.

    
def changingBalance(account: Account) = {
    println("==== Starting balance: " + account.balance)
    println("Depositing $10.0")
    account.deposit(10.0)
    println("new balance: " + account.balance)
    println("Withdrawing $5.60")
    account.withdraw(5.6)
    println("new balance: " + account.balance)
}

var a = new Account(0.0)
changingBalance(a)

var oa = new ObservedAccount(0.0)
changingBalance(oa)
oa.addObserver(new AccountReporter)
changingBalance(oa)

Which prints out:

    
==== Starting balance: 0.0
Depositing $10.0
new balance: 10.0
Withdrawing $5.60
new balance: 4.4
==== Starting balance: 0.0
Depositing $10.0
new balance: 10.0
Withdrawing $5.60
new balance: 4.4
==== Starting balance: 4.4
Depositing $10.0
Observed balance change: 14.4
new balance: 14.4
Withdrawing $5.60
Observed balance change: 8.8
new balance: 8.8

Note that we only observe the last transaction.

Download Scala and try it out. Put all this code in one observer.scala file, for example, and run the command:

    
scala observer.scala

But Wait, There’s More!

In the next post, I’ll look at Scala’s support for Functional Programming and why OO programmers should find it interesting. In the third post, I’ll look at the specific case of concurrent programming in Scala and make some concluding observations of the pros and cons of Scala.

For now, here are some references for more information.

The Scala website, for downloads, documentation, mailing lists, etc.
Ted Neward’s excellent multipart introduction to Scala at developerWorks.
The forthcoming Programming in Scala book.

Always <tt>close()</tt> in a <tt>finally</tt> block

Dean Wampler — Thu, 31 Jul 2008 00:12:00 +0000

Here’s one for my fellow Java programmers, but it’s really generally applicable.

When you call close() on I/O streams, readers, writers, network sockets, database connections, etc., it’s easy to forgot the most appropriate idiom. I just spent a few hours fixing some examples of misuse in otherwise very good Java code.

What’s wrong the following code?

    
public void writeContentToFile(String content, String fileName) throws Exception {
    File output = new File(fileName);
    OutputStreamWriter writer = new OutputStreamWriter(new FileOutputStream(output), "UTF-8");
    writer.write(content);
    writer.close();
}

It doesn’t look all that bad. It tells it’s story. It’s easy to understand.

However, it’s quite likely that you won’t get to the last line, which closes the writer, from time to time. File and network I/O errors are common. For example, what if you can’t actually write to the location specified by fileName? So, we have to be more defensive. We want to be sure we always clean up.

The correct idiom is to use a try … finally … block.

    
public void writeContentToFile(String content, String fileName) throws Exception {
    File output = new File(getFileSystemPath() + contentFilename);
    OutputStreamWriter writer = null;
    try {
        writer = new OutputStreamWriter(new FileOutputStream(output), "UTF-8");
        writer.write(content);
    } finally {
        if (writer != null)
            writer.close();
    }
}

Now, no matter what happens, the writer will be closed, if it’s not null, even if writing the output was unsuccessful.

Note that we don’t necessarily need a catch block, because in this case we’re willing to let any Exceptions propagate up the stack (notice the throws clause). A lot of developers don’t realize that there are times when you need a try block, but not necessarily a catch block. This is one of those times.

So, anytime you need to clean up or otherwise release resources, use a finally block to ensure that the clean up happens, no matter what.

Contracts and Integration Tests for Component Interfaces

Dean Wampler — Sun, 29 Jun 2008 21:54:00 +0000

I am mentoring a team that is transitioning to XP, the first team in a planned, corporate-wide transition. Recently we ran into miscommunication problems about an interface we are providing to another team.

The problems didn’t surface until a “big-bang” integration right before a major release, when it was too late to fix the problem. The feature was backed out of the release, as a result.

There are several lessons to take away from this experience and a few techniques for preventing these problems in the first place.

End-to-end automated integration tests are a well-established way of catching these problems early on. The team I’m mentoring has set up its own continuous-integration (CI) server and the team is getting pretty good at writing acceptance tests using FitNesse. However, these tests only cover the components provided by the team, not the true end-to-end user stories. So, they are imperfect as both acceptance tests and integration tests. Our longer-term goal is to automate true end-to-end acceptance and integration tests, across all components and services.

In this particular case, the other team is following a waterfall-style of development, with big design up front. Therefore, my team needed to give them an interface to design against, before we were ready to actually implement the service.

There are a couple of problems with this approach. First, the two teams should really “pair” to work out the interface and behavior across their components. As I said, we’re just starting to go Agile, but my goal is to have virtual feature teams, where members of the required component teams come together as needed to implement features. This would help prevent the miscommunication of one team defining an interface and sharing it with another team through documentation, etc. Getting people to communicate face-to-face and to write code together would minimize miscommunication.

Second, defining a service interface without the implementation is risky, because it’s very likely you will miss important details. The best way to work out the details of the interface is to test drive it in some way.

This suggests another technique I want to introduce to the team. When defining an interface for external consumption, don’t just deliver the “static” interface (source files, documentation, etc.), also deliver working Mock Objects that the other team can test against. You should develop these mocks as you test drive the interface, even if you aren’t yet working on the full implementation (for schedule or other reasons).

The mocks encapsulate and enforce the behavioral contract of the interface. Design by Contract is a very effective way of thinking about interface design and implementing automated enforcement of it. Test-driven development mostly serves the same practical function, but thinking in “contractual” terms brings clarity to tests that is often missing in many of the tests I see.

Many developers already use mocks for components that don’t exist yet and find that the mocks help them design the interfaces to those components, even while the mocks are being used to test clients of the components.

Of course, there is no guarantee that the mocks faithfully represent the actual behavior, but they will minimize surprises. Whether you have mocks or not, there is no substitute for running automated integration tests on real components as soon as possible.

So... You want your code to be maintainable.

Uncle Bob — Tue, 24 Jun 2008 23:07:46 +0000

We know that maintenance is 90% of the software lifecycle, and 90% of the cost. We know that our systems need to be flexible, reusable, and maintainable. Indeed, that’s why we spend so much of our time trying to get the design and architecture just right. Because we all know that good design and architecture is the key to flexibility, reusability, and maintainability…right?

Of course. Good design and architecture is what makes software easy to change. Good design and architecture separates the things that change for one reason from the things that change for another reason (The Single Responsibility Principle). Good design allows us to add new features without changing a lot of old code (Open Closed Principle). Good design makes sure that high level policy does not depend on low level detail (Dependency Inversion Principle), etc. etc.

So how do we get good design? Well, that’s tricky. Oh it’s not too tricky to get a good design in place at first. The tricky part is to keep the design good. That’s the problem, you see. It’s not that the design starts out so bad (although sometimes…) rather it is that the design degrades over time as the system changes.

Systems change. Often they change in ways that thwart the original intent of the design. Unfortunately, changing the design to align to these changes is hard. So we wind up hacking the new features into the system and thwarting the design. And that’s how even the best designed systems rot.

So how do we keep the design from rotting? How do we make sure we can migrate the design as the system changes? Simple. Tests.

When you have a suite of tests that covers >90% of the code in the system, you are not afraid to make changes. Every time you make a little change you run those tests, and you know that you have not broken anything. This gives you the confidence to make the next change, and the next, and the next. It gives you the confidence to change the design!

Nothing makes a system more flexible than a suite of tests. Nothing. Good architecture and design are important; but the affect of a robust suite of tests is an order of magnitude greater. It’s so much greater because those tests enable you to improve the design.

This can’t be overstated. If you want your systems to be flexible, write tests. If you want your systems to be reusable, write tests. If you want your systems to be maintainable, write tests.

And write your tests using the Three Laws of TDD.

Observations on Test-Driving User Interfaces

Dean Wampler — Sun, 22 Jun 2008 16:52:00 +0000

Test driving user interface development has always been a challenge. Recently, I’ve worked with two projects where most of the work has been on the user-interface components.

The first project is using Adobe Flex to create a rich interface. The team decided to adopt FunFX for acceptance testing. You write your tests in Ruby, typically using Test::Unit or RSpec.

FunFX places some constraints on your Flex application. You have to define the GUI objects in MXML, the XML-based file format for Flex applications, rather than ActionScript, and you need to add ids to all elements you want to reference.[1]

These are reasonable constraints and the first constraint promotes better quality, in fact. The MXML format is more succinct (despite the XML “noise”) and declarative than ActionScript code. This is almost always true of UI code in most languages (with notable exceptions…). Declarative vs. imperative code tends to improve quality because less code means fewer bugs, less to maintain, and it frees the implementor of the declarative “language” to pick the best implementation strategies, optimizations, etc. This characteristic is typical of Functional Languages and well-designed Domain Specific Languages, as well.

I don’t think you can underestimate the benefit of writing less code. I see too many teams whose problems would diminish considerably if they just got rid of duplication and learned to be concise.

The second project is a wiki-based application written in Java. To make deployment as simple as possible, the implementors avoided the Servlet API (no need to install Tomcat, etc.) and rolled their own web server and page rendering components. (I’m not sure I would have made these decisions myself, but I don’t think they are bad, either…)

The rendering components are object-oriented and use a number of design patterns, such as page factories with builder objects that reflect the “widgets” in the UI, HTML tags, etc. This approach makes the UI very testable with JUnit and FitNesse. In fact, the development process was a model of test-driven development.

However, the final result is flawed! It is much too difficult to change the look and feel of the application, which is essential for most UI’s, especially web UI’s. The project made the wrong tradeoffs; the design choices met the requirements of TDD very well, but they made maintenance and enhancement expensive and tedious. The application is now several years old and it has become dated, because of the expense of “refreshing” the look and feel.

What should have been done? These days, most dynamic web UI’s are built with templating engines, of which there are many in the most common programming languages. Pages defined in a templating engine are very declarative, except for the special tags where behavior is inserted. The pages are easy to change. It is mostly obvious where a particular visual element is generated, since most of the “tags” in the template look exactly like the tags in the rendered page. “Declarative” templates, like good DSL’s, can be read, understood, and even edited by the stakeholders, in this case the graphical designers.

But how do you test these page templates? When test-driving UI’s it is important to decide what to test and what not to test. The general rule for TDD is to test anything that can break. The corollary, especially relevant for UI’s, is don’t test anything when you don’t care if it changes.

It is usually the dynamic behavior of the UI that can break and should be tested. Templating engines provide special tags for inserting dynamic behavior in the underlying language (Java, Ruby, etc.). This is what you should test. It is usually best to keep the scripts in these tags as small as possible; the scripts just delegate to code, which can be test-driven in the usual way.

I see too many UI tests that compare long strings of HTML. These tests break whenever someone makes a minor look and feel or other inconsequential change. Part of the art of UI TDD is knowing how to test just what can break and nothing more. In the second project, incidental changes to the UI break tests that should be agnostic to such changes.

To conclude, keep your UI’s as declarative as you can. Only test the “declarations” (e.g., templates) in areas where they might break, meaning if it changes, it’s a bug. You’ll get the full benefits of TDD and the freedom to change the UI easily and frequently, as needed.

¹ Disclaimer: my information on FunFX is second hand, so I might not have the details exactly correct; see the FunFX documentation for details.

What We Can Learn from the Ojibwe Language

Dean Wampler — Sat, 03 May 2008 09:48:57 +0000

Ojibwe (sometimes spelled Ojibwa; the last syllable is pronounced “way”) is one of the few Native American languages that isn’t immediately threatened with extinction. It is spoken by about 10,000 people around the Great Lakes region. Brothers David and Anton Treuer are helping to keep it alive, as they discussed in a recent Fresh Air interview.

Ojibwe is a language that is optimized for an aboriginal people whose lives and livelihoods depend on an intimate awareness of their environment, especially the weather and water conditions. They have many nouns and verbs for fine gradations of rain, snow, ice conditions, the way water waves look and sound, etc. You would want this clarity of detail if you ventured out on a lake every day to fish for dinner.

In the past, speaking languages like Ojibwe was actively suppressed by the government, in an attempt to assimilate Native Americans. Today, the threat of extinction is more from the sheer ubiquity of English. I think there is another force at play, too. People living a modern, so-called “developed” lifestyle just don’t need to be so aware of their environment anymore. In fact, most of us are pretty “tone deaf” to the nuances of weather and water, which is sad in a way. We just don’t perceive the need for the richness of an Ojibwe to communicate what’s important to us, like sports trivia and fashion tips.

So, what does Ojibwe have to do with programming languages? Our language choices inform the way we frame problem solving and design. I was reminded of this recently while reading Ted Neward’s series of articles on Scala. Scala is a JVM language that provides first-class support for functional programming and object-oriented design refinements like traits, which provide mixin behavior.

While you can write Java-like code in Scala, Neward demonstrates how exploiting Scala features can result in very different code for many problems. The Scala examples are simpler, but sometimes that simplicity only becomes apparent after you grasp the underlying design principle in use, like closures or functional idioms.

One of the best pieces of advice in the Pragmatic Programmer is to learn a new language every year. You should pick a language that is very different from what you know already, not one that is fundamentally similar. Even if you won’t use that language in your professional work, understanding its principles, patterns, and idioms will inform your work in whatever languages you actually use.

For example, there is a lot of fretting these days about concurrent programming, given the rise of multi-core CPUs and multiprocessor computers. We know how to write concurrent programs in our most popular imperative languages, like Java and C++, but that knowledge is somewhat specialized and not widely known in the community. This is the main reason that functional programming is suddenly interesting again. It is inherently easier to write concurrent applications using side-effect-free code. I expect that we will fail to meet the concurrency challenge if we rely exclusively on the mechanisms in our imperative languages.

So, you could adopt a functional language for all or part of your concurrent application. Or, if you can’t use Scala (or Haskell or Erlang or …) you could at least apply functional idioms, like side-effect-free functions, avoidance of mutable objects, etc. in your current imperative language. However, even that won’t be an option unless you understand those principles in the first place.

Learning a new language is more than learning a new vocabulary. It’s even more than learning new design techniques. It’s also learning to see common things from a fresh perspective, with greater clarity.

The Post-itÂ® Notes Test for UML Diagrams

Dean Wampler — Tue, 22 Jan 2008 15:58:00 +0000

A lot of teams require their developers to document their designs in UML, using Visio or another tool, before they can start coding.

Of course, this is not at all Agile. For one thing, the design is likely to change quite a bit as you learn while coding. Hardly anyone returns to the diagrams and updates them. Now they are lies, because they make claims about the designs that aren’t true.

UML still has a place in agile projects, of course. It’s a great tool for brainstorming design ideas. So, how do you decide when a diagram is worth keeping and therefore, worth maintaining? Here’s a little strategy that I recommend.

Draw the diagram during those brainstorming sessions on a white board or a poster-sized Post-it® Note. Drawing it this way means you have invested almost no additional effort, beyond the brainstorming itself, to create the diagram. Also, you won’t feel bad about lost work if you eventually throw it away.

Leave the diagram on the wall for everyone to see while they implement the design.

By the time the note is falling off the wall or the dry-erase marker is wearing off the white board, you’ll know if the ideas are still relevant or completely obsolete.

If they are obsolete, you can erase the board or toss the paper. If they are still relevant, and probably changed somewhat, you now know that the diagram is worth preserving. Go ahead and spend the time to create an updated, more permanent version in your drawing tool (but don’t spend too much time!).

TDD for AspectJ Aspects

Dean Wampler — Tue, 02 Oct 2007 11:34:24 +0000

There was a query on the TDD mailing list about how to test drive aspects. Here is an edited version of my reply to that list.

Just as for regular classes, TDD can drive aspects to a better design.

Assume that I’m testing a logging aspect that logs when certain methods are called. Here’s the JUnit 4 test:

package logging;
import static org.junit.Assert.*;
import org.junit.Test;
import app.TestApp;

public class LoggerTest {
    @Test
    public void FakeLoggerShouldBeCalledForAllMethodsOnTestClasses() {
        String message = "hello!";
        new TestApp().doFirst(message);
        assertTrue(FakeLogger.messageReceived().contains(message));
        String message2 = "World!";
        new TestApp().doSecond(message, message2);
        assertTrue(FakeLogger.messageReceived().contains(message));
    }
}

Already, you might guess that FakeLogger will be a test-only version of something, in this case, my logging aspect. Similarly, TestApp is a simple class that I’m using only for testing. You might choose to use one or more production classes, though.

package app;
@Watchable
public class TestApp {
    public void doFirst(String message) {}
    public void doSecond(String message1, String message2) {}
}

and @Watchable is a marker annotation that allows me to define pointcuts in my logger aspect without fragile coupling to concrete names, etc. You could also use an interface.

package app;
public @interface Watchable {}

I made up @Watchable as a way of marking classes where the public methods might be of “interest” to particular observers of some kind. It’s analogous to the EJB 3 annotations that mark classes as “persistable” without implying too many details of what that might mean.

Now, the actual logging is divided into an abstract base aspect and a test-only concrete sub-aspect>

package logging;

import org.aspectj.lang.JoinPoint;
import app.Watchable;

abstract public aspect AbstractLogger {
    // Limit the scope to the packages and types you care about.
    public abstract pointcut scope();

    // Define how messages are actually logged.
    public abstract void logMessage(String message);

    // Notice the coupling is to the @Watchable abstraction.
    pointcut watch(Object object):
        scope() && call(* (@Watchable *).*(..)) && target(object);

    before(Object watchable): watch(watchable) {
        logMessage(makeLogMessage(thisJoinPoint));
    }

    public static String makeLogMessage(JoinPoint joinPoint) {
        StringBuffer buff = new StringBuffer();
        buff.append(joinPoint.toString()).append(", args = ");
        for (Object arg: joinPoint.getArgs())
            buff.append(arg.toString()).append(", ");
        return buff.toString();
    }
}

and

package logging;

public aspect FakeLogger extends AbstractLogger {
    // Only match on calls from the unit tests.
    public pointcut scope(): within(logging.*Test);

    public void logMessage(String message) {
        lastMessage = message; 
    }

    static String lastMessage = null;
    public static String messageReceived() {
        return lastMessage;
    }
}

Pointcuts in aspects are like most other dependencies, best avoided ;) ... or at least minimized and based on abstractions, just like associations and inheritance relationships.

So, my test “pressure” drove the design in terms of where I needed abstraction in the Logger aspect: (i) how a message is actually logged and (ii) what classes get “advised” with logging behavior.

Just as for TDD of regular classes, the design ends up with minimized dependencies and flexibility (abstraction) where it’s most useful.

I can now implement the real, concrete logger, which will also be a sub-aspect of AbstractLogger. It will define the scope() pointcut to be a larger section of the system and it will send the message to the real logging subsystem.

ANN: OOPSLA Tutorial on "Principles of Aspect-Oriented Design in Java and AspectJ"

Dean Wampler — Thu, 13 Sep 2007 11:34:29 +0000

I’m doing a tutorial on aspect-oriented design principles with examples in Java and AspectJ at OOPSLA this year (October 21st). You can find a description here. I believe Friday, 9/14, is the last day for early, discounted registration, so sign up now!

A short presentation on the same subject can be found here.

Are "else" blocks the root of all evil?

Dean Wampler — Tue, 05 Jun 2007 14:42:00 +0000

So, I’m pair programming C++ code with a client today and he makes an observation that makes me pause.

The well-structured, open-source code I’ve looked at typically has very few else blocks. You might see a conditional test with a return statement if the conditional evaluates to true, but not many if/else blocks.

(I’m quoting from memory…) Now, this may seem crazy at first, but one of the principles we teach at Object Mentor is the Single Responsibility Principle, which states that a class should have only one reason to change. This principle also applies to methods. More loosely defined, a class or method should do only one thing.

So, if a method has an if/else block, is it doing two (or more) things and therefore violating the SRP?

Okay, so this is a bit too restrictive (and the title was a bit of an attention grabber… ;). We’re not talking about something really evil, like premature optimization, after all!

However, look at your own if/else blocks and ask yourself if maybe your code would express its intent better if you refactored it to eliminate some of those else blocks.

So, is there something to this idea?

AOP and Dynamic Languages: Contradiction in Terms or Match Made in Heaven?

Dean Wampler — Wed, 21 Mar 2007 13:21:35 +0000

Consider this quote from Dave Thomas (of the Pragmatic Programmers) on AOP (aspect-oriented programming, a.k.a. aspect-oriented software development - AOSD) :

Once you have decent reflection and metaclasses, AOP is just part of the language.

People who work with dynamic languages don't see any need for AOP-specific facilities in their language. They don't necessarily dispute the value of AOP for Java, where metaprogramming facilities are weaker, but for them, AOP is just a constrained form of metaprogramming. Are they right?

It's easy to see why people feel this way when you consider that most of the applications of AOP have been to solve obvious "cross-cutting concerns" (CCCs) like object-relational mapping, transactions, security, etc. In other words, AOP looks like just one of many tools in your toolbox to solve a particular group of problems.

I'll use Ruby as my example dynamic language, since Ruby is the example I know best. It's interesting to look at Ruby on Rails source code, where you find a lot of "AOP-like" code that addresses the CCCs I just mentioned (and more). This is easy enough to do using Ruby's metaprogramming tools, even though tooling that supports AOP semantics would probably make this code easier to write and maintain.

This is going to be a long blog entry already, so I won't cite detailed examples here, but consider how Rails uses method_missing to effectively "introduce" new methods into classes and modules. For example, in ActiveRecord, the many possible find methods and attribute read/write methods are "implemented" this way.

By the way, another excellent Ruby framework, RSpec used method_missing for similar purposes, but recently refactored its implementation and public API to avoid method_missing, because having multiple frameworks attempt to use the same "hook" proved very fragile!

Also in Rails, method "aliasing" is done approximately 175 times, often to wrap ("advise") methods with new behaviors.

Still, is there really a need for AOP tooling in dynamic languages? First, consider that in the early days of OOP, some of us "faked" OOP using whatever constructs our languages provided. I wrote plenty of C code that used structs as objects and method pointers to simulate method overloading and overriding. However, few people would argue today that such an approach is "good enough". If we're thinking in objects, it sure helps to have a language that matches those semantics.

Similarly, it's true that you can implement AOP using sufficiently powerful metaprogramming facilities, but it's a lot harder than having native AOP semantics in your language (or at least a close approximation thereof in libraries and their DSLs).

Before proceeding, let me remind you what AOP is for in the first place. AOP is essentially a new approach to modularization that complements other approaches, like objects. It tries to solve a group of problems that other modularity approaches can't handle, namely the fine-grained interaction of multiple domain models that is required to implement required functionality.

Take the classic example of security management. Presumably, you have one strategy and implementation for handling authentication and authorization. This is one domain and your application's "core business logic" is another domain.

In a non-AOP system, it is necessary to insert duplicate or nearly duplicate code throughout the system that invokes the security subsystem. This duplication violates DRY, it clutters the logic of the code where it is inserted, it is difficult to test, maintain, replace with a new implementation, etc.

Now you may say that you handle this through a Spring XML configuration file or an EJB deployment configuration file, for example. Congratulations, you are using an AOP or AOP-like system!

What AOP seeks to do is to allow you to specify that repeated behavior in one place, in a modular way.

There are four pieces required for an AOP system:

1. Interception

You need to be able to "intercept" execution points in the program. We call these join points in the AOP jargon and sets of them that the aspect writer wants to work with at once are called pointcuts (yes, no whitespace). At each join point, advice is the executable code that an aspect invokes either before, after or both before and after ("around") the join point.

Note that the most powerful AOP language, AspectJ, let's you advise join points like instance variable reads and writes, class initialization, instance creation, etc. The easiest join points to advise are method calls and many AOP systems limit themselves to this capability.

2. Introduction

Introduction is the ability to add new state and behavior to an existing class, object, etc. For example, if you want to use the Observer pattern with a particular class, you could use an aspect to introduce the logic to maintain the list of observers and to notify them when state changes occur.

3. Inspection

We need to be able to find the join points of interest, either through static or runtime analysis, preferably both! You would also like to specify certain conditions of interest, which I'll discuss shortly.

4. Modularization

If we can't package all this into a "module", then we don't have a new modularization scheme. Note that a part of this modularization is the ability to somehow specify in one place the behavior I want and have it affect the entire system. Hence, AOP is a modularity system with nonlocal effects.

Okay. How does pure Ruby stack up these requirements? If you're a Java programmer, the idea of Interception and Introduction, where you add new state and behavior to a class, may seem radical. In languages with "open classes" like Ruby, it is trivial and common to reopen a class (or Module) and insert new attributes (state) and methods (behavior). You can even change previously defined methods. Hence, Interception and Introduction are trivial in Ruby.

This is why Ruby programmers assume that AOP is nothing special, but what they are missing are the complete picture for Inspection and Modularization, even though both are partially covered.

There is a rich reflection API for finding classes and objects. You can write straightforward code that searches for classes that "respond to" a particular method, for example. What you can't do easily is query based on state. For example, in AspectJ, you can say, "I want to advise method call X.m when it is called in the context flow ('cflow') of method call Y.m2 somewhere up the stack..." Yes, you can figure out how to do this in Ruby, but it's hard. So, we're back to the argument I made earlier that you would really like your language to match the semantics of your ideas.

For modularization, yes you can put all the aspect-like code in a Module or Class. The hard part is encapsulating any complicated "point cut" metaprogramming in one place, should you want to use it again later. That is, once you figure out how to do the cflow pointcuts using metaprogramming, you'll want that tricky bit of code in a library somewhere.

At this point, you might be saying to yourself, "Okay, so it might be nice to have some AOP stuff in Ruby, but the Rails guys seem to be doing okay without it. Is it really worth the trouble having AOP in the language?" Only if AOP is more applicable than for the limited set of problems described previously.

Future Applications of AOP??

Here's what I've been thinking about lately. Ruby is a wonderful language for creating mini-DSLs. The ActiveRecord DSL is a good example. It provides relational semantics, while the library minimizes the coding required by the developer. (AR reads the database schema and builds an in-memory representation of the records as objects.)

Similarly, there is a lot of emphasis these days on development that centers around the domain or features of the project. Recall that I said that AOP is about modularizing the intersection of multiple domains (and recall my previous blog on the AOSD 2007 Conference where Gerald Sussman remarked that successful systems have more than one organizing principle).

I think we'll see AOP become the underlying implementation of powerful DSLs that allow programmers who are not AOP-literate express cross-cutting concerns in domain-specific and intuitive languages. AOP will do the heavy lifting behind the scenes to make the fine-grained interactions work. I really don't expect a majority of developers to become AOP literate any time soon. In my experience, too many so-called developers don't get objects. They'll never master aspects!

Shameless Plug

If you would like to hear more of my ideas about AOP in Ruby and aspect-oriented design (AOD), please come to my talk at SD West, this Friday at 3:30. I'm also giving a full-day tutorial on AOD in Ruby and Java/AspectJ at the ICSE 2007 conference in Minneapolis, May 26th.

Liskov Substitution Principle and the Ruby Core Libraries

Dean Wampler — Sat, 17 Feb 2007 14:20:00 +0000

There is a spirited discussion happening now on the ruby-talk list called >Oppinions on RCR for dup on immutable classes (sic).

In the core Ruby classes, the Kernel module, which is the root of everything, even Object, defines a method called dup, for duplicating objects. (There is also a clone method with slightly different behavior that I won’t discuss here.)

The problem is that some derived core classes throw an exception when dup is called.

Specifically, as the ruby-talk discussion title says, it’s the immutable classes (NilClass, FalseClass, TrueClass, Fixnum, and Symbol) that do this. Consider, for example, the following irb session:

irb 1:0> 5.respond_to? :dup
=> true
irb 2:0> 5.dup
TypeError: can't dup Fixnum
        from (irb):1:in `dup'
        from (irb):1
irb 3:0>

If you don’t know Ruby, the first line asks the Fixnum object 5 if it responds to the method dup (with the name expressed as a symbol, hence the). The answer is true, becuase this method is defined by the module Kernel, which is included by the top-level class Object, an ancestor of Fixnum.

However, when you actually call dup on 5, it raises TypeError, as shown.

So, this looks like a classic Liskov Substitution Principle violation. The term for this code smell is Refused Bequest (e.g., see here) and it’s typically fixed with the refactoring Replace Inheritance with Delegation.

The email thread is about a proposal to change the library in one of several possible ways. One possibility is to remove dup from the immutable classes. This would eliminate the unexpected behavior in the example above, since 5.respond_to?(:dup) would return false, but it would still be an LSP violation, specifically it would still have the Refused Bequest smell.

One scenario where the current behavior causes problems is doing a deep copy of an arbitrary object graph. For immutable objects, you would normally just want dup to return the same object. It’s immutable, right? Well, not exactly, since you can re-open classes and even objects to add and remove methods in Ruby (there are some limitations for the immutables…). So, if you thought you actually duplicated something and started messing with its methods, you would be surprised to find the original was “also” modified.

So, how serious is this LSP issue (one of several)? When I pointed out the problem in the discussion, one respondent, Robert Dober, said the following (edited slightly):

I would say that LSP does not apply here simply because in Ruby we do not have that kind of contract. In order to apply LSP we need to say at a point we have an object of class Base, for example. (let the gods forgive me that I use Java)

void aMethod(final Base b){
   ....
}

and we expect this to work whenever we call aMethod with an object that is a Base. Anyway the compiler would not really allow otherwise.

SubClass sc;  // subclassing Base od course
aMethod( sc ); // this is expected to work (from the type POV).

Such things just do not exist in Ruby, I believe that Ruby has explained something to me:

OO Languages are Class oriented languages

Dynamic Languages are Object oriented languages.

Replace Class with Type and you see what I mean.

This is all very much IMHO of course but I feel that the Ruby community has made me evolve a lot away from “Class oriented”.

He’s wrong that the compiler protects you in Java; you can still throw exceptions, etc. The JDK Collection classes have Refused Bequests. Besides that, however, he makes some interesting points.

As a long-time Java programmer, I’m instinctively uncomfortable with LSP violations. Yet, the Ruby API is very nice to work with, so maybe a little LSP violation isn’t so bad?

As Robert says, we approach our designs differently in dynamic vs. static languages. In Ruby, you almost never use the is_a? and kind_of? methods to check for type. Instead, you follow the duck typing philosophy (“If it acts like a duck, it must be a duck”); you rely on respond_to? to decide if an object does what you want.

In the case of dup for the immutable classes, I would prefer that they not implement the method, rather than throw an exception. However, that would still violate LSP.

So, can we still satisfy LSP and also have rich base classes and modules?

There are many examples of traits that one object might or should support, but not another. (Those of you Java programmers might ask yourself why all objects support toString, for example. Why not also toXML...?)

Coming from an AOP background, I would rather see an architecture where dup is added only to those classes and modules that can support it. It shouldn’t be part of the standard “signature” of Kernel, but it should be present when code actually needs it.

In fact, Ruby makes this sort of AOP easy to implement. Maybe Kernel, Module, and Object should be refactored into smaller pieces and programmers should declaratively mixin the traits they need. Imagine something like the following:

irb 1:0> my_obj.respond_to? :dup
=> false
irb 2:0> include 'DupableTrait'  
irb 2:0> my_obj.respond_to? :dup
=> true
irb 4:0> def dup_if_possible items
irb 5:1>  items.map {|item| item.respond_to?(:dup) ? item.dup : item}
irb 6:1> end
...

In other words, Kernel no longer “exposes the dup abstraction”, by default, but the DupableTrait module “magically” adds dup to all the classes that can support it. This way, we preserve LSP, streamline the core classes and modules (SRP and ISP anyone?), yet we have the flexibility we need, on demand.