Object Mentor Blog: Category Uncle Bob's Blatherings

Clean Code. Whew!

Uncle Bob — Tue, 08 Apr 2008 00:10:16 -0500

I’ve been working on this book for several years now. After a flurry of effort (you might have noticed I’ve been quiet lately) I’m very pleased to say that I’m done with the writing and am preparing the manuscript for production. See The Prentice Hall Listing

Table of Contents

Clean Code    1
    There Will Be Code    1
    Bad Code    2
    The Total Cost of Owning a Mess.    3
    Schools of Thought.    11
    We are Authors.    12
    The Boy Scout Rule    13
    Prequel and Principles    14
    Conclusion    14
    Bibliography    15
Meaningful Names by Tim Ottinger    17
    Introduction    17
    Use Intention-revealing Names    17
    Avoid Disinformation    19
    Make Meaningful Distinctions    20
    Use Pronounceable Names    21
    Use Searchable Names    22
    Avoid Encodings    23
    Avoid Mental Mapping    24
    Class Names    25
    Method Names    25
    Don't Be Cute    25
    Pick One Word Per Concept    26
    Don't Pun    26
    Use Solution Domain Names    27
    Use Problem Domain Names    27
    Add Meaningful Context    27
    Don't add Gratuitous Context    29
    Final Words ...    30
Functions    31
    Small!    34
    Do one thing.    35
    One level of abstraction per function.    36
    Switch Statements.    37
    Use descriptive names.    39
    Function Arguments.    39
    Have no side-effects.    43
    Command Query Separation    44
    Prefer exceptions to returning error codes.    45
    Don't Repeat Yourself.    47
    Structured Programming    48
    How do you write functions like this?    48
    Conclusion    49
    SetupTeardownIncluder    49
    Bibliography    52
Comments    53
    Comments do not make up for bad code.    55
    Explain yourself in code.    55
    Good Comments    55
    Bad Comments    59
    Example    71
    Bibliography    74
Formatting    75
    The Purpose of Formatting    76
    Vertical Formatting    76
    Horizontal Formatting    84
    Team Rules    89
    Uncle Bob's Formatting Rules.    90
Objects and Data Structures    93
    Data Abstraction    93
    Data/Object anti-symmetry.    95
    The Law of Demeter    97
    Data Transfer Objects    99
    Conclusion    101
    Bibliography    101
Error Handling by Michael Feathers    103
    Use Exceptions Rather than Return Codes    103
    Write Your Try-Catch-Finally Statement First    105
    Use Unchecked Exceptions    106
    Provide Context with Exceptions    107
    Define Exception Classes In Terms of a Caller's Needs.    107
    Define the Normal Flow    109
    Don't Return Null    110
    Don't Pass Null    111
    Conclusion    112
    Bibliography    112
Boundaries by James Grenning    113
    Bibliography    119
Unit Tests    121
    The Three Laws of TDD    122
    Keeping Tests Clean    123
    Clean Tests    124
    One Assert per Test    129
    F.I.R.S.T.    132
    Conclusion    132
    Bibliography    133
Classes    135
    Class Organization    135
    Classes should be Small!    136
    Organizing for Change    146
    Bibliography    150
Systems    By Dean Wampler 151
    How would you build a city?    151
    Separate constructing a system from using it    152
    Scaling Up    155
    Java Proxies    158
    Pure Java AOP Frameworks    160
    AspectJ Aspects    163
    Test-drive the system architecture    164
    Optimize decision making    165
    Use standards wisely, when they add demonstrable value    165
    Systems need Domain-Specific Languages    166
    Conclusion    166
    Bibliography    167
Emergence By Jeff Langr    169
    Getting Clean via Emergent Design    169
    Simple Design Rule 1: Runs all the tests    170
    Simple Design Rules 2-4: Refactoring    170
    No Duplication    170
    Expressive    173
    Minimal Classes and Methods    174
    Conclusion    174
    Bibliography    174
Concurrency    by Brett Schuchert 175
    Why Concurrency?    176
    Challenges    177
    Concurrency Defense Principles    178
    Know Your Library    180
    Know Your Execution Models    181
    Beware Dependencies between Syncrhonized Methods    182
    Keep Synchronized Sections Small    183
    Writing Correct Shut-Down Code is Hard    183
    Testing Threaded Code    184
    Conclusion    188
    Bibliography    189
Successive Refinement    191
    Args Implementation    192
    Args: the rough draft.    198
    String Arguments    212
     Conclusion    246
JUnit Internals    249
    Conclusion    262
Refactoring SerialDate    263
    Conclusion    280
    Bibliography    281
Smells and Heuristics    283
    Comments    283
    Environment    284
    Functions    285
    General    285
    Java    304
    Names    306
    Tests    310
    Conclusion    311
    Bibliography    312
Concurrency II    by Brett Schuchert 313
    Client/Server Example    313
    Possible Paths of Execution    317
    Knowing Your Library    322
    Dependencies between methods can break concurrent code    325
    Increasing Throughput    329
    Deadlock    331
    Testing Multi-Threaded Code    335
    Tool Support for Testing Thread-Based Code    337
    Conclusion    338
    Tutorial: Full Code Examples    339
org.jfree.date.SerialDate    345
Cross References of Heuristics    406

Clean Code. Whew!

Uncle Bob — Tue, 08 Apr 2008 00:10:16 -0500

Table of Contents

Clean Code    1
    There Will Be Code    1
    Bad Code    2
    The Total Cost of Owning a Mess.    3
    Schools of Thought.    11
    We are Authors.    12
    The Boy Scout Rule    13
    Prequel and Principles    14
    Conclusion    14
    Bibliography    15
Meaningful Names by Tim Ottinger    17
    Introduction    17
    Use Intention-revealing Names    17
    Avoid Disinformation    19
    Make Meaningful Distinctions    20
    Use Pronounceable Names    21
    Use Searchable Names    22
    Avoid Encodings    23
    Avoid Mental Mapping    24
    Class Names    25
    Method Names    25
    Don't Be Cute    25
    Pick One Word Per Concept    26
    Don't Pun    26
    Use Solution Domain Names    27
    Use Problem Domain Names    27
    Add Meaningful Context    27
    Don't add Gratuitous Context    29
    Final Words ...    30
Functions    31
    Small!    34
    Do one thing.    35
    One level of abstraction per function.    36
    Switch Statements.    37
    Use descriptive names.    39
    Function Arguments.    39
    Have no side-effects.    43
    Command Query Separation    44
    Prefer exceptions to returning error codes.    45
    Don't Repeat Yourself.    47
    Structured Programming    48
    How do you write functions like this?    48
    Conclusion    49
    SetupTeardownIncluder    49
    Bibliography    52
Comments    53
    Comments do not make up for bad code.    55
    Explain yourself in code.    55
    Good Comments    55
    Bad Comments    59
    Example    71
    Bibliography    74
Formatting    75
    The Purpose of Formatting    76
    Vertical Formatting    76
    Horizontal Formatting    84
    Team Rules    89
    Uncle Bob's Formatting Rules.    90
Objects and Data Structures    93
    Data Abstraction    93
    Data/Object anti-symmetry.    95
    The Law of Demeter    97
    Data Transfer Objects    99
    Conclusion    101
    Bibliography    101
Error Handling by Michael Feathers    103
    Use Exceptions Rather than Return Codes    103
    Write Your Try-Catch-Finally Statement First    105
    Use Unchecked Exceptions    106
    Provide Context with Exceptions    107
    Define Exception Classes In Terms of a Caller's Needs.    107
    Define the Normal Flow    109
    Don't Return Null    110
    Don't Pass Null    111
    Conclusion    112
    Bibliography    112
Boundaries by James Grenning    113
    Bibliography    119
Unit Tests    121
    The Three Laws of TDD    122
    Keeping Tests Clean    123
    Clean Tests    124
    One Assert per Test    129
    F.I.R.S.T.    132
    Conclusion    132
    Bibliography    133
Classes    135
    Class Organization    135
    Classes should be Small!    136
    Organizing for Change    146
    Bibliography    150
Systems    By Dean Wampler 151
    How would you build a city?    151
    Separate constructing a system from using it    152
    Scaling Up    155
    Java Proxies    158
    Pure Java AOP Frameworks    160
    AspectJ Aspects    163
    Test-drive the system architecture    164
    Optimize decision making    165
    Use standards wisely, when they add demonstrable value    165
    Systems need Domain-Specific Languages    166
    Conclusion    166
    Bibliography    167
Emergence By Jeff Langr    169
    Getting Clean via Emergent Design    169
    Simple Design Rule 1: Runs all the tests    170
    Simple Design Rules 2-4: Refactoring    170
    No Duplication    170
    Expressive    173
    Minimal Classes and Methods    174
    Conclusion    174
    Bibliography    174
Concurrency    by Brett Schuchert 175
    Why Concurrency?    176
    Challenges    177
    Concurrency Defense Principles    178
    Know Your Library    180
    Know Your Execution Models    181
    Beware Dependencies between Syncrhonized Methods    182
    Keep Synchronized Sections Small    183
    Writing Correct Shut-Down Code is Hard    183
    Testing Threaded Code    184
    Conclusion    188
    Bibliography    189
Successive Refinement    191
    Args Implementation    192
    Args: the rough draft.    198
    String Arguments    212
     Conclusion    246
JUnit Internals    249
    Conclusion    262
Refactoring SerialDate    263
    Conclusion    280
    Bibliography    281
Smells and Heuristics    283
    Comments    283
    Environment    284
    Functions    285
    General    285
    Java    304
    Names    306
    Tests    310
    Conclusion    311
    Bibliography    312
Concurrency II    by Brett Schuchert 313
    Client/Server Example    313
    Possible Paths of Execution    317
    Knowing Your Library    322
    Dependencies between methods can break concurrent code    325
    Increasing Throughput    329
    Deadlock    331
    Testing Multi-Threaded Code    335
    Tool Support for Testing Thread-Based Code    337
    Conclusion    338
    Tutorial: Full Code Examples    339
org.jfree.date.SerialDate    345
Cross References of Heuristics    406

Clean Code. Whew!

Uncle Bob — Tue, 08 Apr 2008 00:10:16 -0500

Table of Contents

Clean Code    1
    There Will Be Code    1
    Bad Code    2
    The Total Cost of Owning a Mess.    3
    Schools of Thought.    11
    We are Authors.    12
    The Boy Scout Rule    13
    Prequel and Principles    14
    Conclusion    14
    Bibliography    15
Meaningful Names by Tim Ottinger    17
    Introduction    17
    Use Intention-revealing Names    17
    Avoid Disinformation    19
    Make Meaningful Distinctions    20
    Use Pronounceable Names    21
    Use Searchable Names    22
    Avoid Encodings    23
    Avoid Mental Mapping    24
    Class Names    25
    Method Names    25
    Don't Be Cute    25
    Pick One Word Per Concept    26
    Don't Pun    26
    Use Solution Domain Names    27
    Use Problem Domain Names    27
    Add Meaningful Context    27
    Don't add Gratuitous Context    29
    Final Words ...    30
Functions    31
    Small!    34
    Do one thing.    35
    One level of abstraction per function.    36
    Switch Statements.    37
    Use descriptive names.    39
    Function Arguments.    39
    Have no side-effects.    43
    Command Query Separation    44
    Prefer exceptions to returning error codes.    45
    Don't Repeat Yourself.    47
    Structured Programming    48
    How do you write functions like this?    48
    Conclusion    49
    SetupTeardownIncluder    49
    Bibliography    52
Comments    53
    Comments do not make up for bad code.    55
    Explain yourself in code.    55
    Good Comments    55
    Bad Comments    59
    Example    71
    Bibliography    74
Formatting    75
    The Purpose of Formatting    76
    Vertical Formatting    76
    Horizontal Formatting    84
    Team Rules    89
    Uncle Bob's Formatting Rules.    90
Objects and Data Structures    93
    Data Abstraction    93
    Data/Object anti-symmetry.    95
    The Law of Demeter    97
    Data Transfer Objects    99
    Conclusion    101
    Bibliography    101
Error Handling by Michael Feathers    103
    Use Exceptions Rather than Return Codes    103
    Write Your Try-Catch-Finally Statement First    105
    Use Unchecked Exceptions    106
    Provide Context with Exceptions    107
    Define Exception Classes In Terms of a Caller's Needs.    107
    Define the Normal Flow    109
    Don't Return Null    110
    Don't Pass Null    111
    Conclusion    112
    Bibliography    112
Boundaries by James Grenning    113
    Bibliography    119
Unit Tests    121
    The Three Laws of TDD    122
    Keeping Tests Clean    123
    Clean Tests    124
    One Assert per Test    129
    F.I.R.S.T.    132
    Conclusion    132
    Bibliography    133
Classes    135
    Class Organization    135
    Classes should be Small!    136
    Organizing for Change    146
    Bibliography    150
Systems    By Dean Wampler 151
    How would you build a city?    151
    Separate constructing a system from using it    152
    Scaling Up    155
    Java Proxies    158
    Pure Java AOP Frameworks    160
    AspectJ Aspects    163
    Test-drive the system architecture    164
    Optimize decision making    165
    Use standards wisely, when they add demonstrable value    165
    Systems need Domain-Specific Languages    166
    Conclusion    166
    Bibliography    167
Emergence By Jeff Langr    169
    Getting Clean via Emergent Design    169
    Simple Design Rule 1: Runs all the tests    170
    Simple Design Rules 2-4: Refactoring    170
    No Duplication    170
    Expressive    173
    Minimal Classes and Methods    174
    Conclusion    174
    Bibliography    174
Concurrency    by Brett Schuchert 175
    Why Concurrency?    176
    Challenges    177
    Concurrency Defense Principles    178
    Know Your Library    180
    Know Your Execution Models    181
    Beware Dependencies between Syncrhonized Methods    182
    Keep Synchronized Sections Small    183
    Writing Correct Shut-Down Code is Hard    183
    Testing Threaded Code    184
    Conclusion    188
    Bibliography    189
Successive Refinement    191
    Args Implementation    192
    Args: the rough draft.    198
    String Arguments    212
     Conclusion    246
JUnit Internals    249
    Conclusion    262
Refactoring SerialDate    263
    Conclusion    280
    Bibliography    281
Smells and Heuristics    283
    Comments    283
    Environment    284
    Functions    285
    General    285
    Java    304
    Names    306
    Tests    310
    Conclusion    311
    Bibliography    312
Concurrency II    by Brett Schuchert 313
    Client/Server Example    313
    Possible Paths of Execution    317
    Knowing Your Library    322
    Dependencies between methods can break concurrent code    325
    Increasing Throughput    329
    Deadlock    331
    Testing Multi-Threaded Code    335
    Tool Support for Testing Thread-Based Code    337
    Conclusion    338
    Tutorial: Full Code Examples    339
org.jfree.date.SerialDate    345
Cross References of Heuristics    406

Clean Code. Whew!

Uncle Bob — Tue, 08 Apr 2008 00:10:16 -0500

Table of Contents

Clean Code    1
    There Will Be Code    1
    Bad Code    2
    The Total Cost of Owning a Mess.    3
    Schools of Thought.    11
    We are Authors.    12
    The Boy Scout Rule    13
    Prequel and Principles    14
    Conclusion    14
    Bibliography    15
Meaningful Names by Tim Ottinger    17
    Introduction    17
    Use Intention-revealing Names    17
    Avoid Disinformation    19
    Make Meaningful Distinctions    20
    Use Pronounceable Names    21
    Use Searchable Names    22
    Avoid Encodings    23
    Avoid Mental Mapping    24
    Class Names    25
    Method Names    25
    Don't Be Cute    25
    Pick One Word Per Concept    26
    Don't Pun    26
    Use Solution Domain Names    27
    Use Problem Domain Names    27
    Add Meaningful Context    27
    Don't add Gratuitous Context    29
    Final Words ...    30
Functions    31
    Small!    34
    Do one thing.    35
    One level of abstraction per function.    36
    Switch Statements.    37
    Use descriptive names.    39
    Function Arguments.    39
    Have no side-effects.    43
    Command Query Separation    44
    Prefer exceptions to returning error codes.    45
    Don't Repeat Yourself.    47
    Structured Programming    48
    How do you write functions like this?    48
    Conclusion    49
    SetupTeardownIncluder    49
    Bibliography    52
Comments    53
    Comments do not make up for bad code.    55
    Explain yourself in code.    55
    Good Comments    55
    Bad Comments    59
    Example    71
    Bibliography    74
Formatting    75
    The Purpose of Formatting    76
    Vertical Formatting    76
    Horizontal Formatting    84
    Team Rules    89
    Uncle Bob's Formatting Rules.    90
Objects and Data Structures    93
    Data Abstraction    93
    Data/Object anti-symmetry.    95
    The Law of Demeter    97
    Data Transfer Objects    99
    Conclusion    101
    Bibliography    101
Error Handling by Michael Feathers    103
    Use Exceptions Rather than Return Codes    103
    Write Your Try-Catch-Finally Statement First    105
    Use Unchecked Exceptions    106
    Provide Context with Exceptions    107
    Define Exception Classes In Terms of a Caller's Needs.    107
    Define the Normal Flow    109
    Don't Return Null    110
    Don't Pass Null    111
    Conclusion    112
    Bibliography    112
Boundaries by James Grenning    113
    Bibliography    119
Unit Tests    121
    The Three Laws of TDD    122
    Keeping Tests Clean    123
    Clean Tests    124
    One Assert per Test    129
    F.I.R.S.T.    132
    Conclusion    132
    Bibliography    133
Classes    135
    Class Organization    135
    Classes should be Small!    136
    Organizing for Change    146
    Bibliography    150
Systems    By Dean Wampler 151
    How would you build a city?    151
    Separate constructing a system from using it    152
    Scaling Up    155
    Java Proxies    158
    Pure Java AOP Frameworks    160
    AspectJ Aspects    163
    Test-drive the system architecture    164
    Optimize decision making    165
    Use standards wisely, when they add demonstrable value    165
    Systems need Domain-Specific Languages    166
    Conclusion    166
    Bibliography    167
Emergence By Jeff Langr    169
    Getting Clean via Emergent Design    169
    Simple Design Rule 1: Runs all the tests    170
    Simple Design Rules 2-4: Refactoring    170
    No Duplication    170
    Expressive    173
    Minimal Classes and Methods    174
    Conclusion    174
    Bibliography    174
Concurrency    by Brett Schuchert 175
    Why Concurrency?    176
    Challenges    177
    Concurrency Defense Principles    178
    Know Your Library    180
    Know Your Execution Models    181
    Beware Dependencies between Syncrhonized Methods    182
    Keep Synchronized Sections Small    183
    Writing Correct Shut-Down Code is Hard    183
    Testing Threaded Code    184
    Conclusion    188
    Bibliography    189
Successive Refinement    191
    Args Implementation    192
    Args: the rough draft.    198
    String Arguments    212
     Conclusion    246
JUnit Internals    249
    Conclusion    262
Refactoring SerialDate    263
    Conclusion    280
    Bibliography    281
Smells and Heuristics    283
    Comments    283
    Environment    284
    Functions    285
    General    285
    Java    304
    Names    306
    Tests    310
    Conclusion    311
    Bibliography    312
Concurrency II    by Brett Schuchert 313
    Client/Server Example    313
    Possible Paths of Execution    317
    Knowing Your Library    322
    Dependencies between methods can break concurrent code    325
    Increasing Throughput    329
    Deadlock    331
    Testing Multi-Threaded Code    335
    Tool Support for Testing Thread-Based Code    337
    Conclusion    338
    Tutorial: Full Code Examples    339
org.jfree.date.SerialDate    345
Cross References of Heuristics    406

The Quality of TDD

Uncle Bob — Sat, 16 Feb 2008 20:23:44 -0600

Kieth Braithwaite has made an interesting observation here. The basic idea is that code that has been written with TDD has a lower Cyclomatic Complexity per function compared to code that has not been written with TDD. If this is true then it could imply lower defects because of this.

Kieth’s metric takes in the code for an entire project and boils it down to a single number. His hypothesis is that a system written with TDD will always measure above a certain threshold, indicating very low CC; whereas systems written without TDD may or may not measure above that threshold.

Kieth has built a tool that you can get here that will generate this metric for most java projects. He and others have used this tool to measure many different systems. So far the hypothesis seems to hold water.

The metric can’t tell you if TDD was used; but it might just be able to tell you that it wasn’t used.

Generated Tests and TDD

Uncle Bob — Thu, 10 Jan 2008 13:59:30 -0600

TDD has become quite popular, and many companies are attempting to adopt it. However, some folks worry that it takes a long time to write all those unit tests and are looking to test-generation tools as a way to decrease that burden.

The burden is not insignificant. FitNesse, an application created using TDD, is comprised of 45,000 lines of Java code, 15,000 of which are unit tests. Simple math suggests that TDD increases the coding burden by a full third!

Of course this is a naive analysis. The benefits of using TDD are significant, and far outweigh the burden of writing the extra code. But that 33% still feels “extra” and tempts people to find ways to shrink it without losing any of the benefits.

Test Generators.

Some folks have put their hope in tools that automatically generate tests by inspecting code. These tools are very clever. They generate random calls to methods and remember the results. They can automatically build mocks and stubs to break the dependencies between modules. They use remarkably clever algorithms to choose their random test data. They even provide ways for programmers to write plugins that adjust those algorithms to be a better fit for their applications.

The end result of running such a tool is a set of observations. The tool observes how the instance variables of a class change when calls are made to its methods with certain arguments. It notes the return values, changes to instance variables, and outgoing calls to stubs and mocks. And it presents these observations to the user.

The user must look through these observations and determine which are correct, which are irrelevant, and which are bugs. Once the bugs are fixed, these observations can be checked over and over again by re-running the tests. This is very similar to the record-playback model used by GUI testers. Once you have registered all the correct observations, you can play the tests back and make sure those observations are still being observed.

Some of the tools will even write the observations as JUnit tests, so that you can run them as a standard test suite. Just like TDD, right? Well, not so fast…

Make no mistake, tools like this can be very useful. If you have a wad of untested legacy code, then generating a suite of JUnit tests that verifies some portion of the behavior of that code can be a great boon!

The Periphery Problem

On the other hand, no matter how clever the test generator is, the tests it generates will always be more naive than the tests that a human can write. As a simple example of this, I have tried to generate tests for the bowling game program using two of the better known test generation tools. The interface to the Bowling Game looks like this:

  public class BowlingGame {
    public void roll(int pins) {...}
    public int score() {...}
  }

The idea is that you call roll each time the balls gets rolled, and you call score at the end of the game to get the score for that game.

The test generators could not randomly generate valid games. It’s not hard to see why. A valid game is a sequence of between 12 and 21 rolls, all of which must be integers between 0 and 10. What’s more, within a given frame, the sum of rolls cannot exceed 10. These constraints are just too tight for a random generator to achieve within the current age of the universe.

I could have written a plugin that guided the generator to create valid games; but such an algorithm would embody much of the logic of the BowlingGame itself, so it’s not clear that the economics are advantageous.

To generalize this, the test generators have trouble getting inside algorithms that have any kind of protocol, calling sequence, or state semantics. They can generate tests around the periphery of the classes; but can’t get into the guts without help.

TDD?

The real question is whether or not such generated tests help you with Test Driven Development. TDD is the act of using tests as a way to drive the development of the system. You write unit test code first, and then you write the application code that makes that code pass. Clearly generating tests from existing code violates that simple rule. So in some philosophical sense, using test generators is not TDD. But who cares so long as the tests get written, right? Well, hang on…

One of the reasons that TDD works so well is that it is similar to the accounting practice of dual entry bookkeeping. Accountants make every entry twice; once on the credit side, and once on the debit side. These two entries follow separate mathematical pathways. In the end a magical subtraction yields a zero if all the entries were made correctly.

In TDD, programmers state their intent twice; once in the test code, and again in the production code. These two statements of intent verify each other. The tests, test the intent of the code, and the code tests the intent of the tests. This works because it is a human that makes both entries! The human must state the intent twice, but in two complementary forms. This vastly reduces many kinds of errors; as well as providing significant insight into improved design.

Using a test generator breaks this concept because the generator writes the test using the production code as input. The generated test is not a human restatement, it is an automatic translation. The human states intent only once, and therefore does not gain insights from restatement, nor does the generated test check that the intent of the code was achieved. It is true that the human must verify the observations, but compared to TDD that is a far more passive action, providing far less insight into defects, design and intent.

I conclude from this that automated test generation is neither equivalent to TDD, nor is it a way to make TDD more efficient. What you gain by trying to generate the 33% test code, you lose in defect elimination, restatement of intent, and design insight. You also sacrifice depth of test coverage, because of the periphery problem.

This does not mean that test generators aren’t useful. As I said earlier, I think they can help to partially characterize a large base of legacy code. But these tools are not TDD tools. The tests they generate are not equivalent to tests written using TDD. And many of the benefits of TDD are not achieved through test generation.

Business software is Messy and Ugly.

Uncle Bob — Thu, 13 Dec 2007 09:41:00 -0600

I was at a client recently. They are a successful startup who have gone through a huge growth spurt. Their software grew rapidly, through a significant hack-and-slash program. Now they have a mess, and it is slowing them way down. Defects are high. Unintended consequences of change are high. Productivity is low.

I spent two days advising them how to adopt TDD and Clean Code techniques to improve their code-base and their situation. We discussed strategies for gradual clean up, and the notion that big refactoring projects and big redesign projects have a high risk of failure. We talked about ways to clean things up over time, while incrementally insinuating tests into the existing code base.

During the sessions they told me of a software manager who is famed for having said:

“There’s a clean way to do this, and a quick-and-dirty way to do this. I want you to do it the quick-and-dirty way.”

The attitude engendered by this statement has spread throughout the company and has become a significant part of their culture. If hack-and-slash is what management wants, then that’s what they get! I spent a long time with these folks countering that attitude and trying to engender an attitude of craftsmanship and professionalism.

The developers responded to my message with enthusiasm. They want to do a good job (of course!) They just didn’t know they were authorized to do good work. They thought they had to make messes. But I told them that the only way to get things done quickly, and keep getting things done quickly, is to create the cleanest code they can, to work as well as possible, and keep the quality very high. I told them that quick-and-dirty is an oxymoron. Dirty always means slow.

On the last day of my visit the infamous manager (now the CTO) stopped into our conference room. We talked over the issues. He was constantly trying to find a quick way out. He was manipulative and cajoling. “What if we did this?” or “What if we did that?” He’d set up straw man after straw man, trying to convince his folks that there was a time and place for good code, but this was not it.

I wanted to hit him.

Then he made the dumbest, most profoundly irresponsible statement I’ve (all too often) heard come out of a CTOs mouth. He said:

“Business software is messy and ugly.”

No, it’s not! The rules can be complicated, arbitrary, and ad-hoc; but the code does not need to be messy and ugly. Indeed, the more arbitrary, complex, and ad-hoc the business rules are, the cleaner the code needs to be. You cannot manage the mess of the rules if they are contained by another mess! The only way to get a handle on the messy rules is to express them in the cleanest and clearest code you can.

In the end, he backed down. At least while I was there. But I have no doubt he’ll continue his manipulations. I hope the developers have the will to resist.

One of the developers asked the question point blank: “What do you do when your managers tell you to make a mess?” I responded: “You don’t take it. Behave like a doctor who’s hospital administrator has just told him that hand-washing is too expensive, and he should stop doing it.”

Thinking about an Appendix.

Uncle Bob — Fri, 07 Dec 2007 09:27:04 -0600

No, not of Clean Code (look for it in Spring). This monday evening (12/3) I got a stomach ache.

The stomach pain was a low level annoyance that I was able to ignore for most of the evening. But it started to get pretty bad round midnight. By 2AM I threw up and then felt better. So I went back to bed and slept till morning.

Tuesday started well. I had some residual pain, but figured it was waning. I went to the office to get some work done there. By the time I got there the waning pain had waxed and I turned right around and went home to bed.

The pain continued to grow until 3PM when I thew up again. This made me feel a little better so I went back to bed. But by 6PM the pain had localized to my lower right quadrant and I had had enough. I asked my wife to take me to the immediate care facility.

They could tell by the grimace on my face that this was something to take care of quickly. They put me on a bed, and started an IV. They took blood and did an exam. Then they packed me into an ambulance and sent me to the ER for a CAT scan.

During the 20 minute ride the pain was getting really bad. I was giving it 8.5 on a scale of 1-10. I have intimate knowledge of every pothole on the road the took.

At the ER they put me in a room and gave me a dose of Morphine. Morphine is a very nice drug. It had the effect of filing the pain away in a convenient subdirectory where I could access it if I needed it, but was otherwise out of the way.

By 11PM the CAT scan was complete and the doctor came and said: “The good news is we know what’s wrong. The bad news is it’s your appendix. We just happen to have an OR ready. Do you want to do it now?” How could I refuse an offer like that!

The rolled me into the OR and told me they were going to give me the juice.

DISCONTINUITY

I was in the recovery room with two smiling pretty nurses urging me to awaken to the news that my appendix had been removed.

DISCONTINUITY

I was being wheeled into a hospital room. It was 3AM. My exhausted wife was waiting for me there. She kissed me goodbye and I went to sleep.

Wednesday was my birthday. I spent is sleeping for the most part. They fed me clear foods. The pain was not horrible, but I did accept a vicodin around noon. By 4PM the surgeon came in, looked me over, and said: “You look pretty good. You should go home.” I went home gladly. On the way home my wife and I picked up a pizza.

Thursday was a bit sore, but not bad. I took a couple of motrin and was “smart” about what muscles I used and when. I got a lot of work-reading done.

Today I woke up and the incision pain was like a mild sunburn. I can feel it, but I can ignore it. I’m not about to do 20 pushups, or run a marathon, but the day should be relatively normal.

Active Record vs Objects

Uncle Bob — Fri, 02 Nov 2007 11:29:31 -0500

Active Record is a well known data persistence pattern. It has been adopted by Rails, Hibernate, and many other ORM tools. It has proven it’s usefulness over and over again. And yet I have a philosophical problem with it.

The Active Record pattern is a way to map database rows to objects. For example, let’s say we have an Employee object with name and address fields:

public class Employee extends ActiveRecord {
  private String name;
  private String address;
  ...
}

We should be able to fetch a given employee from the database by using a call like:

Employee bob = Employee.findByName("Bob Martin");

We should also be able to modify that employee and save it as follows:

bob.setName("Robert C. Martin");
bob.save();

In short, every column of the Employee table becomes a field of the Employee class. There are static methods (or some magical reflection) on the ActiveRecord class that allow you to find instances. There are also methods that provide CRUD functions.

Even shorter: There is a 1:1 correspondence between tables and classes, columns and fields. (Or very nearly so).

It is this 1:1 correspondence that bothers me. Indeed, it bothers me about all ORM tools. Why? Because this mapping presumes that tables and objects are isomorphic.

The Difference between Objects and Data Structures

From the beginning of OO we learned that the data in an object should be hidden, and the public interface should be methods. In other words: objects export behavior, not data. An object has hidden data and exposed behavior.

Data structures, on the other hand, have exposed data, and no behavior. In languages like C++ and C# the struct keyword is used to describe a data structure with public fields. If there are any methods, they are typically navigational. They don’t contain business rules.

Thus, data structures and objects are diametrically opposed. They are virtual opposites. One exposes behavior and hides data, the other exposes data and has no behavior. But that’s not the only thing that is opposite about them.

Algorithms that deal with objects have the luxury of not needing to know the kind of object they are dealing with. The old example: shape.draw(); makes the point. The caller has no idea what kind of shape is being drawn. Indeed, if I add new types of shapes, the algorithms that call draw() are not aware of the change, and do not need to be rebuilt, retested, or redeployed. In short, algorithms that employ objects are immune to the addition of new types.

By the same token, if I add new methods to the shape class, then all derivatives of shape must be modified. So objects are not immune to the addition of new functions.

Now consider an algorithm that uses a data structure.


switch(s.type) {
  case SQUARE: Shape.drawSquare((Square)s); break;
  case CIRCLE: Shape.drawCircle((Circle)s); break;
}

We usually sneer at code like this because it is not OO. But that disparagement might be a bit over-confident. Consider what happens if we add a new set of functions, such as Shape.eraseXXX(). None of the existing code is effected. Indeed, it does not need to be recompiled, retested, or redeployed. Algorithms that use data structures are immune to the addition of new functions.

By the same token if I add a new type of shape, I must find every algorithm and add the new shape to the corresponding switch statement. So algorithms that employ data structures are not immune to the addition of new types.

Again, note the almost diametrical opposition. Objects and Data structures convey nearly opposite immunities and vulnerabilities.

Good designers uses this opposition to construct systems that are appropriately immune to the various forces that impinge upon them. Those portions of the system that are likely to be subject to new types, should be oriented around objects. On the other hand, any part of the system that is likely to need new functions ought to be oriented around data structures. Indeed, much of good design is about how to mix and match the different vulnerabilities and immunities of the different styles.

Active Record Confusion

The problem I have with Active Record is that it creates confusion about these two very different styles of programming. A database table is a data structure. It has exposed data and no behavior. But an Active Record appears to be an object. It has “hidden” data, and exposed behavior. I put the word “hidden” in quotes because the data is, in fact, not hidden. Almost all ActiveRecord derivatives export the database columns through accessors and mutators. Indeed, the Active Record is meant to be used like a data structure.

On the other hand, many people put business rule methods in their Active Record classes; which makes them appear to be objects. This leads to a dilemma. On which side of the line does the Active Record really fall? Is it an object? Or is it a data structure?

This dilemma is the basis for the oft-cited impedance mismatch between relational databases and object oriented languages. Tables are data structures, not classes. Objects are encapsulated behavior, not database rows.

At this point you might be saying: “So what Uncle Bob? Active Record works great. So what’s the problem if I mix data structures and objects?” Good question.

Missed Opportunity

The problem is that Active Records are data structures. Putting business rule methods in them doesn’t turn them into true objects. In the end, the algorithms that employ Active Records are vulnerable to changes in schema, and changes in type. They are not immune to changes in type, the way algorithms that use objects are.

You can prove this to yourself by realizing how difficult it is to implement an polymorphic hierarchy in a relational database. It’s not impossible of course, but every trick for doing it is a hack. The end result is that few database schemae, and therefore few uses of Active Record, employ the kind of polymorphism that conveys the immunity of changes to type.

So applications built around ActiveRecord are applications built around data structures. And applications that are built around data structures are procedural—they are not object oriented. The opportunity we miss when we structure our applications around Active Record is the opportunity to use object oriented design.

No, I haven’t gone off the deep end.

I am not recommending against the use of Active Record. As I said in the first part of this blog I think the pattern is very useful. What I am advocating is a separation between the application and Active Record.

Active Record belongs in the layer that separates the database from the application. It makes a very convenient halfway-house between the hard data structures of database tables, and the behavior exposing objects in the application.

Applications should be designed and structured around objects, not data structures. Those objects should expose business behaviors, and hide any vestige of the database. The fact that we have Employee tables in the database, does not mean that we must have Employee classes in the application proper. We may have Active Records that hold Employee rows in the database interface layer, but by the time that information gets to the application, it may be in very different kinds of objects.

Conclusion

So, in the end, I am not against the use of Active Record. I just don’t want Active Record to be the organizing principle of the application. It makes a fine transport mechanism between the database and the application; but I don’t want the application knowing about Active Records. I want the application oriented around objects that expose behavior and hide data. I generally want the application immune to type changes; and I want to structure the application so that new features can be added by adding new types. (See: The Open Closed Principle)

A Rational Plan to Start using SOA

Uncle Bob — Sun, 21 Oct 2007 05:00:09 -0500

Many companies are thinking about adopting SOA as a way to reduce the cost of enterprise development. The thought of diverse and reusable services, effortlessly communicating through an intelligent Enterprise Service Bus is a powerful siren song for executives who see their development and maintenance budgets spiraling upwards. What’s more, vendors are displaying savory looking wares that promise to ease the transition.

Caveat Emptor!

The reality is that cramming an old and rickety enterprise system into services that communicate over a complex buss, is not an easy job. The long and short of the problem can be seen in the following catch-22:

The benefit of SOA comes from the decoupled nature of the services. But in order to create an SOA you have to decouple your services.

Decoupling your services is the issue. Once you have decoupled them, then all the wonderful tools might be useful in facilitating an SOA. On the other hand, if you have decoupled your services, you probably don’t need all those wonderful tools.

I have a client that has implemented SOA to good effect. They avoided the vast majority of the tools. For example, they are not using a big expensive buss. They aren’t using a BPEL engine, or message translators, or routing services. About the only thing they have decided to use is SOAP.

They are creating their services from scratch, and wiring the system together manually. Each service knows where each of it’s client services is. There is no registry service, no common dictionary, no fancy routing.

The system is simple, elegant, and it works. It was designed to work from the start. It is not an old crufty system crammed under a soap layer and called a service provider. Rather it is a nicely arrayed suite of services that were designed to work together from the get-go.

And that’s really the moral of the story. If you want an SOA, you are going to have to design that SOA. It is not very likely that you will be able to cram you old, tightly-coupled code below SOAP interfaces and hope to gain the benefits of services. What you’ll more than likely end up with is just a bigger mess.

Architecture is a Second Order Effect

Uncle Bob — Sat, 20 Oct 2007 01:58:29 -0500

We often argue that in order to achieve flexibility, maintainability, and reusability, it is important to have a good software architecture. This is certainly true. Without a well ordered architecture, software systems become masses of tangled modules and code. However, the effect of architecture on the ‘ilities is secondary to the effect that a good, fast, suite of tests has.

Why don’t we clean our code? When we see an ugly mass of code that we know is going to cause of problems, our first reaction is “This needs to be cleaned up.” Our second reaction is: “If I touch this code I’ll be spending the next two weeks trying to get it to work again.” We don’t clean code because we are afraid we’ll break it.

In this way bad code hooks itself into our systems and refuses to go away. Nothing stops clean code from going bad, but once it’s bad, we seldom have the time, energy, or nerve to clean it. In that sense, bad code is permanent.

What if you had a button? If you push this button a little light instantly turns red or green. Green means your system works. Red means it’s broken. If you had a button like this, you could make small changes to the system, and prove that they didn’t break anything. If you saw a batch of tangled messy code, you could start to clean it. You’d simply make a tiny improvement and then push the button. If the light was green you’d make the next tiny change, and the next, and the next.

I have a button like that! It’s called a test suite. I can run it any time I like, and within seconds it tells me, with very high certainty, that my system works.

Of course, to be effective, that test suite has to cover all the code in the system. And, indeed, that’s what I have. My code coverage tools tell me that 89% of the 45,000 lines in my system are covered by my test suite. (89% is pretty good number given that my coverage tool counts un-executable lines like interfaces.)

Can I be absolutely sure that the green light means that the system works? Certainly not. But given the coverage of my test suite, I am reasonably sure that the changes I make are not introducing any bugs. And that reasonable surety is enough for me to be fearless about making changes.

This fearlessness is something that needs to be experienced to understand. I feel no reluctance at all about cleaning up the code in my system. I frequently take whole classes and restructure them. I change the names of classes, variables, and functions on a whim. I extract super-classes and derivatives any time I like.

In short, the test suite makes it easy to make changes to my code. It makes my code flexible and easy to maintain.

So does my architecture of course. The design and structure of my system is very easy to deal with, and allows me to make changes without undue impact. The reason my architecture is so friendly, is that I’ve been fearless about changing it. And that’s because I have a test suite that runs in seconds, and that I trust!

So the clean architecture of my system is a result of on-going efforts supported by the test suite. I can keep the architecture clean and relevant because I have tests. I can improve the architecture when I see a better approach because I have tests. It is the tests that enable architectural improvement.

Yes, architecture enables flexibility, maintainability, and reusability; but test suites enable architecture. Architecture is a second order effect.

TDD with Acceptance Tests and Unit Tests

Uncle Bob — Wed, 17 Oct 2007 05:50:34 -0500

Test Driven Development is one of the most imperative tenets of Agile Software Development. It is difficult to claim that you are Agile, if you are not writing lots of automated test cases, and writing them before you write the code that makes them pass.

But there are two different kinds of automated tests recommended by the Agile disciplines. Unit tests, which are written by programmers, for programmers, in a programming language. And acceptance tests, which are written by business people (and QA), for business people, in a high level specification language (like FitNesse www.fitnesse.org).

The question is, how should developers treat these two streams of tests? What is the process? Should they write their unit tests and production code first, and then try to get the acceptance tests to pass? Or should they get the acceptance tests to pass and then backfill with unit tests?

And besides, why do we need two streams of tests. Isn’t all that testing awfully reduntant?

It’s true that the two streams of tests test the same things. Indeed, that’s the point. Unit tests are written by programers to ensure that the code does what they intend it to do. Acceptance tests are written by business people (and QA) to make sure the code does what they intend it to do. The two together make sure that the business people and programmers intend the same thing.

Of course there’s also a difference in level. Unit tests reach deep into the code and test independent units. Indeed, programmers must go to great lengths to decouple the components of the system in order to test them independently. Therefore unit tests seldom exercise large integrated chunks of the system.

Acceptance tests, on the other hand, operate on much larger integrated chunks of the system. They typically drive the system from it’s inputs (or a point very close to it’s inputs) and verify operation from it’s outputs (or again, very close to it’s outputs). So, though the acceptance tests may be testing the same things as the unit tests, the execution pathways are very different.

Process

Acceptance tests should be written at the start of each iteration. QA and Business analysts should take the stories chosen during the planning meeting, and turn them into automated acceptance tests written in FitNesse, or Selenium or some other appropriate automation tool.

The first few acceptance tests should arrive within a day of the planning meeting. More should arrive each day thereafter. They should all be complete by the midpoint of the iteration. If they aren’t, then some developers should change hats and help the business people finish writing the acceptance tests.

Using developers in this way is an automatic safety valve. If it happens too often, then we need to add more QA or BA resources. If it never happens, we may need to add more programmers.

Programmers use the acceptance tests as requirements. They read those tests to find out what their stories are really supposed to do.

Programmers start a story by executing the acceptance tests for that story, and noting what fails. Then they write unit tests that force them to write the code that will make some small portion of the acceptance tests pass. They keep running the acceptance tests to see how much of their story is working, and they keep adding unit tests and production code until all the acceptance tests pass.

At the end of the iteration all the acceptance tests (and unit tests) are passing. There is nothing left for QA to do. There is no hand-off to QA to make sure the system does what it is supposed to. The acceptance tests already prove that the system is working.

This does not mean that QA does not put their hands on the keyboards and their eyes on the screen. They do! But they don’t follow manual test scripts! Rather, they perform exploratory testing. They get creative. They do what QA people are really good at—they find new and interesting ways to break the system. They uncover unspecified, or under-specified areas of the system.

ASIDE: Manual testing is immoral. Not only is it high stress, tedious, and error prone; it’s just wrong to turn humans into machines. If you can write a script for a test procedure, then you can write a program to execute that procedure. That program will be cheaper, faster, and more accurate than a human, and will free the human to do what humans to best: create!

So, in short, the business specifies the system with automated acceptance tests. Programmers run those tests to see what unit tests need to be written. The unit tests force them to write production code that passes both tests. In the end, all the tests pass. In the middle of the iteration, QA changes from writing automated tests, to exploratory testing.

SOA, cuts the Gordian Knot -- Not.

Uncle Bob — Wed, 03 Oct 2007 13:31:27 -0500

In 333 B.C. Alexander the Great cut the Gordian Knot with his sword, breaking that symbol of ancient power and ushering in a new empire. Some IT managers feel that SOA will finally cut the great Gordian Knot of their tangled and tumultuous software systems. Will it?

No.

Let me restate that more clearly for those of you who may still have doubt.

No frickin way Jose!

Most software systems are a mess. SOA is not the cure for that mess. Indeed, SOA doesn’t even start to be feasible until the mess has been cleaned up.

Again, let me restate that.

SOA does not help you clean up the mess that your software is in. In order to adopt SOA you must first clean up the mess you have made. Once the mess is clean, then you can start to think about SOA.

Right now, at this very moment, some IT manager is reading about how the next release of some Enterprise Service Bus is going to solve all his woes. All he has to do is install this wonderful piece of software, tie it into the new whippy-diddle business process modeling tool. Get some clerks to write the XML files that describe their business processes, get some other clerks to write the message translators that drive that great interoperability Babel Fish in the sky. Get some business process modelers to draw pretty pictures of all the interconnections between the systems. Feed that all into the ESB, and shake vigorously with deadlines and threats. This will cause fuzzy bunnies to hop happily over the green fields of his IT systems.

Right.

The reality is that the first letter of SOA, namely “S”, stands for “Service”. Not to put to fine a point on it, SOA means an Architecture Oriented around Services. What is a service? A Service is a piece of software with the characteristics of a service? What are those characteristics? They are: Low coupling, independently deployability, stable interface, etc.

Most software professionals (if that term is not oxymoronic) will recognize this set of attributes as being identical with that of the “Components” of the 90s. Indeed, Services are the new Components. The words are synonyms.

This means that SOA is just as hard to do as COA. (Component Oriented Architecture for those of you who are acronym challenged). Indeed, SOA is COA!

Now, I know it’s been a long time. But does anybody remember what it took to do COA? No? Then let me tell you a little story to jog your memory.

In the mid ‘90s I had a client who had a 250MB executable written in C++. (Yes, I know this seems like a hello world program today, but back then 250MB was a big program). This client was vexed because every time he changed even a single line of code, he had to rebuild the entire executable, burn it onto a CD, and ship the CDs to his installed base. This was costing him lots of time and money.

What he wanted was a way to break his application into components and then ship only those components that changed. His hope was that he could ship them over the internet because they’d all be small enough to email or FTP. (This was back when people did not regularly download 4GB movies, or backup 100GB disks over the internet every night).

So he adopted COA, and the popular implementation platform of for COA, COM+ (Otherwise known as void**: Nothing to the power of nothing.) He slaved away for months breaking his application up into components that could be dynamically loaded in DLLs. And in the end, he succeeded. His application was completely—componentized.

However, there was one little snag. He forgot to decouple the components from one another. Each component depended on several others. There were cycles in the dependency graph of components. (Aside to the architecturally challenged: “That’s bad.”) So in the end, when he changed a line of code in one component, he had to rebuild and redeploy all the components. He had to burn all those components onto a CD.

In short, nothing had changed.

Long long ago, I watched a software developer write complicated macro in his highly powerful text editor. It converted all tabs to spaces. I remember watching him start it. The code he was transforming was on the screen. He ran the macro, which took several minutes to run. It displayed a series of dots running across the screen. It was very impressive, and looked like it was doing something significant. And then when it finished, the code was redisplayed on the screen. I laughed, because after all that work of translating tabs into spaces, and putting dots on the screen, the displayed code looked exactly the same as when he started. It was as if nothing had happened.

My client’s reaction to his inability to independently deploy his components had nothing to do with laughter.

The point? SOA will not cut the Gordian Knot. Indeed, it will pull that Knot even tighter. The Knot can’t be cut. There is no magic that will make the mess go away, or make the mess less of an issue. If you want the flexibility of SOA, then you must clean up the mess you have and then slowly and gradually build services from those cleaned up systems.

The real point? TANSTAASB. (There ain’t no such thing as a Silver Bullet).

Schools of Thought

Uncle Bob — Wed, 03 Oct 2007 12:37:21 -0500

In some sense we software developers are all too tolerant. We know there is no single “right” way to write software, and so we tolerate many different opinions and styles. In the industry at large, this is a good thing. But inside a project it’s chaos.

A project with 10 different developers cannot withstand 10 different styles of coding, 10 different architectural visions, 10 different philosophies of OO design, or 10 different software development methods. To prevent the chaos of anarchy, one style, one vision, one philosophy and one method must be adopted.

The problem is, we know that no one style, vision, philosophy, or method is “right”. This knowledge makes us timid. We tolerate other people’s styles and attitudes rather than insisting on a single style and vision for the project. We feel that we have no right to insist on one particular way of doing things, and so we adopt a politically correct viewpoint regarding software.

Politically correct projects are muddles. They find it difficult to get traction because the team members are all pulling in different directions. This muddling continues until someone puts a stake in the ground and says “We’re doing it this way”, and makes it stick by sheer force of will. Then the team moves on and makes progress.

Was his way right? The question is irrelevant. The stake in the ground provided focus. It eliminated the chaos of too much “rightness”, of too much political correctness.

There are many different schools of martial arts. Karate, Tai Kwon Do, Jiu Jitsu, Judo, etc. Which is right? Of course the question is absurd. Right and wrong are not adjectives that apply to individual martial arts. And yet, each martial art attracts dedicated adherents. The intensity of their devotion to their art is almost religious.

Students of Karate have no rational reason to prefer it to Tai Kwon Do. Indeed, the initial decision to study one vs the other was almost certainly arbitrary. Perhaps there was a Karate studio near their home. Perhaps a friend invited them to a session. Perhaps they saw something on TV, or in the paper. In most cases the initial decision was trivial.

But that’s where the triviality ends. From then on the new student gets immersed into a set of styles, disciplines, philosophies, and methods to the exclusion of all others. There is no political correctness at this level. Teachers tolerate no dissent from their teachings. It’s the Sen Sei’s way, or the highway. Within a dojo, there is a right and a wrong.

At a larger level, everyone knows that no one martial art is clearly better than all the others. There is no “right” school of martial art. And yet, students invest huge efforts into becoming masters of one particular arbitrary discipline.

Why would a student focus on one particular school of martial art to the exclusion of all others for years of his or her life? Simple. To become a master. You can’t become a master of all martial arts at the same time. You have to separate and focus. You have to dedicate yourself to one and learn it inside and out. You have to be politically incorrect. That’s how you gain mastery.

Once a degree of mastery is attained, it becomes beneficial to study other martial arts. A karate black belt will find it beneficial to study Jiu Jitsu, or Akido. The Judo black belt will gain insights into his own art by studying Karate. So, at the level of the master, political correctness and tolerance is re-established.

There are many diverse attitudes about software. For example there are many different kinds of agile methods. Indeed, there are methods that aren’t agile at all. There’s the American vs the European style of OO. There’s functional vs OO vs procedural programming. There are those who practice hungarian notation. There are those who practice test driven development. There are those who believe in C and C++ and those who believe in Ruby. None of these attitudes and preferences are right. None of them are wrong. But without some kind of organizing principle, like a dojo, or a Sen Sei, they are chaos.

There has been a lot of talk about certification lately. Companies need to know if the people they hire are good developers or not. The problem is, there is no standard of certification. There is no right or wrong, and so there’s little to certify. We can certify that someone took a particular course, or that they have certain knowledge of C++ or Java. We can measure someone’s ability to read a book, or write a dumb example program. What we cannot do is certify whether someone is a good developer or not, because we don’t have a definition of “good”.

The different schools of martial arts have solved this problem of certification quite effectively. No school will certify that their student is a master martial artist. But they will certify that they are a master of one particular discipline. You cannot be a general black-belt, but you can be a black-belt in one particular school of Karate.

The award of rank is not based on a casual test, or the taking of a two day course. It is based on months of exposure and participation in practice sessions. It is based on the teacher’s intimate knowledge of the student’s skill. What’s more the teacher’s right to convey rank is based on his teacher’s authority. Thus, there is a chain of trust from teacher to teacher.

I don’t know if a martial-arts ranking system could ever be established in software. The problems seem significant. For example, how would teachers gain the kind of prolonged access they’d need to bestow ranks on students?

However, I do think it’s possible for experienced practitioners to put a stake in the ground and establish a school of software discipline. In effect the practitioner says: “I’ve done it my way, and it worked for me. I’ll teach you my way, perhaps it will work for you.”

Such a school would be precise and exclusive. Students would adopt a certain coding style, a certain philosophy of design, a certain architectural vision, and a certain set of methods and practices. They would adhere to those certain teachings to the exclusion of all others until they mastered them.

Thread Local: A Convenient Abomination.

Uncle Bob — Tue, 04 Sep 2007 10:16:31 -0500

ThreadLocal variables are a wonderfully convenient way to associate data with a given thread. Indeed, frameworks like Hibernate take advantage of this to hold session information. However, the practice depends upon assumption that a thread is equivalent to a unit-of-work. This is a faulty assumption

Thirteen years ago, while working on my first book, Jim Coplien and I were having a debate on the nature of threads and objects. He made a clarifying statement that has stuck with me since. He said: “An object is an abstraction of function. A thread is an abstraction of schedule.”

It has become the norm, in Java applications, to assume that there is a one-to-one correspondence between a thread, and a unit-of-work. This appears to make sense since every Servlet request has it’s own particular thread. Framework authors have built on this assumption by putting unit-of-work related information (e.g. session) into ThreadLocal variables.

Clearly, the more ThreadLocal variables that hold unit-of-work related information, the more that the thread and the unit-of-work are related. While very convenient, the basic assumption is dead wrong.

A unit-of-work is a task to be performed. There is no rule that says that this task must be single threaded. Nor is there any rule that says that the components of the task must run at the same priority. Indeed, it is not uncommon for a task to have a high-priority event driven part, and a low priority compute-bound part.

Consider, for example, a unit-of-work that makes a request from a third-party-service, and then processes the response with a complex number-crunching algorithm. We want the request to run at a high priority so that it is not waiting for other low priority compute-bound tasks to finish. On the other hand, we want the number-cruncher to run at a low priority so that it yields the processor as soon as possible whenever one of the high priority requests needs a few cycles.

Clearly these two tasks form a complete unit-of-work, but just as clearly they should run as two different threads—probably with a queue in between them in a standard producer-consumer model.

Where are the unit-of-work related variables? They can’t be kept in a ThreadLocal since each part of the task runs in a separate thread. They can’t be kept in static variables since there is more than one thread. The answer is that they have to be passed around between the threads as function arguments on the stack, and recorded in the data structured placed on the queue.

So, though convenient, ThreadLocal variables confuse the issue of separating function from schedule. They tempt us to couple function and schedule together. This is unfortunate since the correspondence of function and schedule is weak and accidental.

What we’d really like is to be able to create UnitOfWorkLocal variables.

Craftsman #51: Brown Bag VIII - Ruby Visitor

Uncle Bob — Wed, 08 Aug 2007 06:54:23 -0500

“A new speaker talks to the team and illuminates the differences between Ruby and Java by implementing the Visitor pattern in both languages.”

Get the PDF.

Or to see all the craftsman articles:

go to www.objectmentor.com,
click the resources tab,
click published articles link,
click the Craftsman topic,
scroll down to #51: Brown Bag VIII – Ruby Visitor.

The Founding of the Agile Alliance

Uncle Bob — Tue, 10 Jul 2007 08:43:00 -0500

These are my own personal recollections. I’m probably wrong about some of it. If any of the other folks involved have a clearer memory of the events, please don’t hesitate to comment.

In the spring of 2000 Kent Beck called a meeting at the Rogue River Lodge near his home in Medford, Oregon. He called it the Extreme Programming Leadership Conference. In attendance were: myself, Ron Jeffries, Ken Auer, Martin Fowler, and several others who had been instrumental in getting the XP movement off the ground.

In one of the sessions we discussed the creation of an organization to drive and support the adoption of XP. This was an idea that I argued in favor of. It seemed to me that the industry would benefit from a body who promoted the ideas of lightweight methods like XP. However, the idea was not met with enthusiasm amongst many of the other attendees. The session turned into a collection of painful reminiscences about a previous attempt at creating an organization around the Patterns movement. I came away convinced that a different group of people would have to drive the creation of such an organization.

Apparently I was not alone. Martin Fowler had noticed the arguments I had made in the session, and approached me during a break. We briefly discussed the idea of broadening the scope of the proposed organization to all the various “lightweight methods” such as Scrum and Crystal Light. We felt that the folks who espoused these methods would have a lot to add, and could form the core of a comprehensive movement. Martin and I determined to meet a few weeks later to work out the details.

In the meantime I contacted my friend and advisor Dave Thomas of OTI fame. (We call him “Big Dave” to differentiate him from the Pragmatic Programmer of the same name.) I mentioned the notion of a Lightweight Methods Summit. We put a tentative invitation list together. Dave suggested that we meet in February at his winter home in Anguilla.

Later that fall I met with Martin at a cafe in Chicago. We added to the invitation list and outlined the invitation email. The invitation stated the goal of creating a manifesto describing what all the various lightweight methods had in common. I sent the email that evening.

The Lightweight Process Summit invitation drew an enthusiastic response from the invitees. Alistair Cockburn was especially interested. As it happened, he was on the verge of sending out his own invitation for a similar event. His comment to me was: “Your invitation list was better than mine”. The two invitation lists were merged, and the discussion of logistics began.

Alistair became the de-facto organizer. He suggested that we change the venue from Anquilla to Snowbird in Salt Lake City. Almost all the other invitees agreed because the flights would be easier. Alistair and Jim Highsmith did the leg-work to arrange the rooms, meals, and activities. Things came together rapidly.

The meeting was very well attended by 17 out of ~20 invitees. (See www.agilemanifesto.org to see the list of attendees.) Unfortunately neither Grady Booch nor Big Dave Thomas were able to make it. However, their influence was strongly felt in the subsequent discussions.

I kicked off the meeting with a brief introduction and an appeal to the similarities between all the various lightweight processes. I reaffirmed that the goal of our two day meeting was to create a manifesto that outlined those similarities. This was to be a document, addressed to the industry at large, describing our views about the process of creating useful software.

Martin Fowler and Ward Cunningham quickly became the de-facto facilitators of the meeting. With their help we quickly worked out an agenda for the two days, and a method for making decisions.

It was actually quite a thrill to see all these people with disparate ideas working together so well. I don’t think I’ve ever been to a meeting that stayed on point and met it’s goals with such ease and with so little conflict. It was as though the pieces just somehow fell together.

One of the early discussions was about a name. No one liked the term Lightweight. Several other options were suggested, including Lean, and Adaptive. But the name Agile won the day.

The structure of the manifesto was mutually agreed; I recall that Ward played a big part framing the idea of pairs of relative value, but it may have been Martin and Pragmatic Dave Thomas who came up with the idea. Here’s PragDave’s recollection:

Actually, that was Martin and me, noodling on the whiteboard over lunch. I think we came up with the initial three. The group that then formed took it up to five, and that was then pruned down to four. In fact, Ward took the picture of everyone discussing the idea that now adorns his manifesto site.

By the end of the two days in Snowbird, the manifesto was done, the principles were outlined, and the Agile Alliance had been born. Much more would need to be done before there would be a true legal entity with a board and members, but that’s a story for another time.

For another version of this history see Jim Highsmith’s description at http://www.agilemanifesto.org/history.html

Craftsman #50: Ruby

Uncle Bob — Thu, 14 Jun 2007 23:30:12 -0500

Here is my latest Craftsman article. Aphonse, Jerry, Jasmine, and the crew learn a little bit about Ruby, and fight about it’s benefits and dangers.

Craftsman #50: Ruby

Uncle Bob — Thu, 14 Jun 2007 23:30:12 -0500

Here is my latest Craftsman article. Aphonse, Jerry, Jasmine, and the crew learn a little bit about Ruby, and fight about it’s benefits and dangers.

Craftsman #50: Ruby

Uncle Bob — Thu, 14 Jun 2007 23:30:12 -0500

Here is my latest Craftsman article. Aphonse, Jerry, Jasmine, and the crew learn a little bit about Ruby, and fight about it’s benefits and dangers.

The Hidiocy of XML Languages

Uncle Bob — Thu, 17 May 2007 22:01:22 -0500

I’ve been reading up on some of the newer aspects of SOA, and came across BPEL. It’s another language you are supposed to write in XML. Get ready for a rant.

I’ve had it. What is the matter with these people? How, after all the experience we’ve had with XSLT, Ant, WSDL, etc., etc., could they create YET ANOTHER XML language. Are they dolts? Are they idiots? What gives.

Look, writing in XML is hideous. It’s wordy, it’s error-prone, it’s arcane, it’s redundant, it’s redundant, it’s redundant, it’s… HIDEOUS! To make matters worse, we have been embedding OTHER languages INSIDE this horrible container. EGAD! YIKES! ZOUNDS! FORSOOTH! This is just plain nuts, stupid, idiotic, retarded, poo-poo-headed, silliness!

Haven’t these people heard of economy of expression? Haven’t they heard of YACC? Don’t they know that domain specific languages SHOULD BE LANGUAGES? Don’t they know that languages have specific GRAMMARS?

Besides, have they ever tried to write an interpreter or compiler that uses XML as it’s source? It’s not any easier than writing a YACC parser! Indeed, it’s a lot, lot, harder.

EMBEDDING THE GRAMMAR OF YOUR DSL IN XML IS STUPID!!! DON’T DO IT!!!!!#$#$!!!

I hereby declare a revolt. From now on anyone who considers themselves to be a serious professional must refuse to write another line of XML. When asked, say NO. Instead, write a little YACC grammar that is nice, and small, and translates into that hideous XML. You’ll save yourself GOBS of time if you do! What’s more, if you sell your parser for $5 per download, you’ll probably be able to buy a new boat! Maybe a fleet!

Coding Standards

Uncle Bob — Wed, 18 Apr 2007 08:55:24 -0500

Coding Standards are a good idea. Every team should adopt a coding style and standard, and stick to it. The code produced by that team should have a consistent look and feel that is devoid of individual preferences and fetishes.

Of course this means that the members of the team will have to be mature enough to realize that it doesn’t really matter where they put their braces, or how they adorn their member variables. What matters is that they all use the same conventions.

Consistency

My goal for a good coding standard is to eliminate individual styles in favor of a team style. The code produced by a team should look like the team produced it. I don’t want any code recognizable as Bob’s or Bill’s.

This is not some egalitarian fantasy to hide individuality for the sake of the collective. Rather, it is a raw necessity. We’ve all seen products that look like they were designed by a committee. We’ve all used software products where the look and feel changed depending on which part of the application you were using. The result feels messy, clumsy, inefficient.

Individuals, used to their own particular style, will reformat other people’s code when forced to work on it, further shuffling the patchwork of styles. Over time, as each team member touches different parts of the code, and team members come and go from the team, the code begins to look like a jumbled Rubick’s cube of different styles.

Code is a product, in and of itself. The team producing it needs to take pride in the elegance of it’s structure, and the expressiveness of it’s presentation. This kind of pride is infeasible when the code is crisscrossed with a patchwork of individual styles. Without this pride, there is no drive to keep the overall product clean. Without that cleanliness, messes build up at the boundaries. And, as we all know, messes slow us down, and they spread.

Style not Substance

A good coding standard should be about style and form, not about substance. It should not attempt to legislate good design. It should not, for example, proscribe goto or public variables. Those rules are part of the body of knowledge that all software developers should have, and are not a matter of style.

Coding standards should be about the things that don’t matter. They should be about brace placement, naming conventions, the use of blank lines, indentation levels, etc. A coding standard should describe the way code looks, not the substance the code is made from.

It is important to keep style and substance separate because they matter for different reasons. Issues of style matter only for consistency. It does not matter whether your indent depth is 2 or 4, so long as everyone uses the same depth. It does matter if you use public variables inappropriately.

Oral/Code Tradition

Documents that describe coding standards tend to be useless. They often become a bloated battleground for many different competing ideas. My advice is to avoid writing them.

The real document that describes your coding standard is your code. If you want to know how to name a variable, look at how they are named in the code. If you want to know what the standard indent depth is, look in the code. The code is the living document that describes the coding standard.

Oral tradition plays a role as well; especially when communicating issues of substance vs. style. Teams should make use of code reviews and pair programming to communicate with each other about issues of style and substance. New members of the team should have frequent exposure to the more seasoned members, so that the issues of style and substance are inculcated with moral authority. Nothing is quite as persuasive to a young programmer than pairing with the lead programmer and hearing him say: “We don’t do things that way; we do things this way.”

The Tyranny of Tools

I have seen teams attempt to enforce a style through the use of tools. Some tools are benign and helpful. Many IDEs, for example, allow you to specify things like indent level, brace placement, etc. With a single keystroke you can ensure that a batch of code conforms to the team style. I use tools like this, an depend upon them. I make sure that all the team members set their IDEs to use the conventions.

Other tools can be more intrusive. Some tools can act like compilers, generating errors if the style is not adhered to. Such tools might be useful for an occasional scan of the code; but I think you have to be very careful if you put them into the normal build cycle.

Automatic enforcement is power; and power corrupts. We do not want a well-meaning bureaucrat deciding, one day, to enforce the style that every function argument must have a javadoc comment. This leads to comments of the form: //required comment.

This is not to say that tools like findbugs and checkstyle should be avoided. Indeed, I find them very useful. However, I think they should be run occasionally and manually, not as part of every build. The issues that these tools discover should be dealt with on a case-by-case basis.

Many tools like this allow you to insert special meta-comments that override the warnings. If these tools are placed in the build process; then the code will become cluttered with these meta-comments.

I have a pathological distaste for meta-comments.

Conclusion

Coding style is a matter of team pride and team identity. Teams should be free to adopt their own styles, and to change those styles as the spirit moves them. Each member of the team should follow the team style, and work to ensure that the body of code is a consistent statement of that style. If this sounds too artsy-fartsy, keep in mind that pride of workmanship is a powerful motivator. We want teams to be proud of their creations.

What is SOA, really?

Uncle Bob — Wed, 11 Apr 2007 07:17:00 -0500

The good news is, you probably already know. The bad news is, you probably know too much. This article describes Service Oriented Architecture in a simple and easy to understand way that is devoid of buzzwords and vendor spin. It’s the introduction to SOA that you haven’t been able to find anywhere else.

There are things in a business that don’t change very often. Gas stations in the U.S., for example, still sell gasoline by the gallon. Restaurants still sell meals from a menu. Dentists still sell cleanings every 6 months. Every business has these aspects that don’t change very frequently. They often represent a huge part of the business. We’ll call these things the _core business functions.

There are other things in a business that change very frequently. Prices, tax rates, catalogs, new products, new marketing campaigns, advertising, new business areas, new customer areas, etc. Indeed, businesses must be able to change, and change quickly, in order to survive. And yet, it is vital that those changes do not adversely affect the core business functions.

Software developers have known for years that software that changes frequently should be decoupled from software that changes infrequently. When applied to individual programs and systems this principle is sometimes called The Common Closure Principle. When it is applied to the information management of an enterprise, it is called SOA.

SOA is the practice of sequestering the core business functions into independent services that don’t change frequently. These services are glorified functions that are called by one or more presentation programs. The presentation programs are volatile bits of software that present data to, and accept data from, various users.

To make this clear, imagine an internet store-front. Customers use a browser to talk to the presentation software that displays the store’s website. The presentation software interprets the gestures of the customer and invokes services that do things like acquiring the data for the current catalog, or registering the customer’s order. Note that the services have no idea they are talking to a website. They could just as well be talking to a thick client, or a 3270 green screen. They simply accept and return data in a standard format that the web system happens to be able to use.

That’s really all there is to it. The rest of SOA is just a matter of details. At the highest level, SOA is nothing more (and nothing less) than separating changeable elements from unchangeable elements. But why is this important?

Consider that internet store-front again. It presents the user with a catalog, allows the user to move items into, and out of a shopping cart, and accepts the eventual order. The presentation of these concepts is very volatile. Marketing people are likely to want to change it frequently. For example, they might want to change from a shopping cart metaphor to scrollable receipt on the sidebar. They may wish to present more or less descriptive data in the product list. They may want to experiment with different colors, font-faces, and layouts. Indeed, it’s feasible that they’ll want to try applets, JStart clients, Ajax, and a myriad of other presentation options. But none of this has anything to do with the core business functions encapsulated by the services. Those services that acquire catalogs and register orders remain unchanged despite all the presentation thrashing. That’s why the separation is important. It protects the information processing assets of the business from the constant jitter and spin of the presentation.

But presentation is not the only thing that jitters and spins. So do the business processes. Again, consider our store-front. Perhaps our business has decided to offer fine wines as one of the products it sells. Selling alcohol requires that the age of the customer be verified. Let us say that we have a service that provides this verification. This service must be called for any order that contains alcohol products. The decision to call this service is neither a presentation decision, nor a service decision. Rather it is part of the business process for a particular kind of order. Business processes are volatile and they breed like rabbits. As businesses evolve they add more and more steps and forks to their business processes. The services being used by those processes don’t change much; but the pathways through the processes do. Therefore we want to separate the business process from the services and from the presentation. Smalltalkers had a name for this separation when it appeared in a single program. They called it Model-View-Controller.

Notice that we have yet to mention even one of the plethora of technologies that are so commonly associated with SOA. That’s because SOA is not about any particular technology. Rather it is a design philosophy that decouples well heeled business functions from volatile processes and presentations. It is the MVC of enterprise software.

In my next blog on this topic, we’ll look at the next level of detail in an attempt to understand HOW services can be constructed, and how the decoupling of presentation, process, and functions can be achieved.