The first project is using Adobe Flex to create a rich interface. The team decided to adopt FunFX for acceptance testing. You write your tests in Ruby, typically using Test::Unit or RSpec.

FunFX places some constraints on your Flex application. You have to define the GUI objects in MXML, the XML-based file format for Flex applications, rather than ActionScript, and you need to add ids to all elements you want to reference.[1]

These are reasonable constraints and the first constraint promotes better quality, in fact. The MXML format is more succinct (despite the XML “noise”) and declarative than ActionScript code. This is almost always true of UI code in most languages (with notable exceptions…). Declarative vs. imperative code tends to improve quality because less code means fewer bugs, less to maintain, and it frees the implementor of the declarative “language” to pick the best implementation strategies, optimizations, etc. This characteristic is typical of Functional Languages and well-designed Domain Specific Languages, as well.

I don’t think you can underestimate the benefit of writing less code. I see too many teams whose problems would diminish considerably if they just got rid of duplication and learned to be concise.

The second project is a wiki-based application written in Java. To make deployment as simple as possible, the implementors avoided the Servlet API (no need to install Tomcat, etc.) and rolled their own web server and page rendering components. (I’m not sure I would have made these decisions myself, but I don’t think they are bad, either…)

The rendering components are object-oriented and use a number of design patterns, such as page factories with builder objects that reflect the “widgets” in the UI, HTML tags, etc. This approach makes the UI very testable with JUnit and FitNesse. In fact, the development process was a model of test-driven development.

However, the final result is flawed! It is much too difficult to change the look and feel of the application, which is essential for most UI’s, especially web UI’s. The project made the wrong tradeoffs; the design choices met the requirements of TDD very well, but they made maintenance and enhancement expensive and tedious. The application is now several years old and it has become dated, because of the expense of “refreshing” the look and feel.

What should have been done? These days, most dynamic web UI’s are built with templating engines, of which there are many in the most common programming languages. Pages defined in a templating engine are very declarative, except for the special tags where behavior is inserted. The pages are easy to change. It is mostly obvious where a particular visual element is generated, since most of the “tags” in the template look exactly like the tags in the rendered page. “Declarative” templates, like good DSL’s, can be read, understood, and even edited by the stakeholders, in this case the graphical designers.

But how do you test these page templates? When test-driving UI’s it is important to decide what to test and what not to test. The general rule for TDD is to test anything that can break. The corollary, especially relevant for UI’s, is don’t test anything when you don’t care if it changes.

It is usually the dynamic behavior of the UI that can break and should be tested. Templating engines provide special tags for inserting dynamic behavior in the underlying language (Java, Ruby, etc.). This is what you should test. It is usually best to keep the scripts in these tags as small as possible; the scripts just delegate to code, which can be test-driven in the usual way.

I see too many UI tests that compare long strings of HTML. These tests break whenever someone makes a minor look and feel or other inconsequential change. Part of the art of UI TDD is knowing how to test just what can break and nothing more. In the second project, incidental changes to the UI break tests that should be agnostic to such changes.

To conclude, keep your UI’s as declarative as you can. Only test the “declarations” (e.g., templates) in areas where they might break, meaning if it changes, it’s a bug. You’ll get the full benefits of TDD and the freedom to change the UI easily and frequently, as needed.

¹ Disclaimer: my information on FunFX is second hand, so I might not have the details exactly correct; see the FunFX documentation for details.

Testing JSP pages.

One of the more annoying aspects of working on a web application is that you have to deploy it in order to test it. This doesn’t apply to everything of course; if you are careful with your design you can test the business rules in plain old java objects. You can test database access, interface layers, and stored procedures without the web server running. But testing the GUI – the HTML produced from the JSP files – is very hard to do unless the system has been deployed.

Lot’s of teams fall back on Selenium, Mercury, or other tools that test GUIs through the web server. However, this leads to very fragile tests that break when the format of a page changes, even though it’s content remains unchanged. Other teams solve the fragility problem by using Cactus, or the somewhat more primitive tools in HtmlUnit and HttpUnit, to inspect the generated HTML delivered to them by the running web application. We’ll talk about those techniques in another blog in this series.

In this article I will demonstrate a simple technique for testing JSP pages using JUnit and HtmlUnit, completely outside the container. The advantages of such a technique should be clear.

• You don’t have to have the container running, or even working. You can test your JSPs before you have even chosen a particular webserver.
• You can spin around the edit/compile/test loop much more quickly because you don’t have to redeploy after every edit.
• You can build your JSPs incrementally using Test Driven Development to guide their construction.

The reason that testing JSPs outside the container isn’t more common is because JSPs are designed to run inside the container. The designers never gave a lot of thought to running them outside the container. So the code generated by the JSP compiler depends on facilities supplied by the container. Even the tools that generate the JSP code expect that you already have a webapp that can be deployed. Therefore, in order to run outside the container, you somehow have to fake these facilities and tools.

Dependency Management Rant.

Compiling JSPs

The first step to testing JSPs is to compile them into servlets. To do that, we first have to translate them from JSP format to JAVA. The apache project provides a tool named Jasper that does this. Invoking Jasper on a JSP named MyPage.jsp creates a JAVA source file named MyPage_jsp.java. This file can then be compiled into a servlet using your favorite IDE. (Mine is IntelliJ).

Unfortunately Jasper was not designed to be run from the command line. Or rather, half of it was, and half of it wasn’t. Jasper does in fact have a main function that takes standard command line arguments. And it is feasible to issue the command java org.apache.jasper.JspC to invoke it. However, Jasper expects the environment it runs in to be consistent with the environment of the container. It expects many the apache JAR files to be in the CLASSPATH. It also expects to see a web.xml file that describes the web application, and it wants to see a WEB-INF directory with all the appropriate web application JAR files and TLD files present. In short, Jasper expects there to be a webapp.

To make matters worse, there appear to be bugs in the version of Jasper that I am using (tomcat 5.5.20) that cause it to generate slightly incorrect code unless it is invoked in a manner that is consistent with the way that TOMCAT invokes it.

The first issue is inconvenient but is just a matter of creating the appropriate directories and files and then invoking Jaser from ANT so that the classpaths are easy to control. The second issue required a fair bit of research and experimentation to get working right. But in the end, the following ant file seems to work quite nicely. Look at the last task to see the JspC invocation.

<project name="Library" default="compile" basedir=".">
  <property environment="env"/>
  <property name="build.home" value="${basedir}/build"/>
  <property name="build.war.home" value="${build.home}/war"/>
  <property name="build.classes.home" value="${build.home}/classes"/>
  <property name="build.jar.home" value="${build.home}/jars"/>
  <property name="catalina.home" value="${env.CATALINA_HOME}"/>
  <property name="dist.home" value="${basedir}/dist"/>
  <property name="web.home" value="${basedir}/web"/>

  <path id="compile.classpath">
    <fileset dir="lib">
      <include name="*.jar"/>
    </fileset>
    <pathelement location="${catalina.home}/common/classes"/>
    <fileset dir="${catalina.home}/common/endorsed">
      <include name="*.jar"/>
    </fileset>
    <fileset dir="${catalina.home}/common/lib">
      <include name="*.jar"/>
    </fileset>
    <pathelement location="${catalina.home}/shared/classes"/>
    <fileset dir="${catalina.home}/shared/lib">
      <include name="*.jar"/>
    </fileset>
  </path>

  <target name="clean">
    <delete dir="${build.home}"/>
    <delete dir="${dist.home}"/>
  </target>

  <target name="compile">
    <mkdir dir="${build.classes.home}"/>
    <javac srcdir="${src.home}" destdir="${build.classes.home}" excludes="**/*Test.java">
      <classpath refid="compile.classpath"/>
    </javac>
  </target>

  <target name="jar" depends="compile">
    <mkdir dir="${build.jar.home}"/>
    <jar jarfile="${build.jar.home}/application.jar" basedir="${build.classes.home}" includes="**/application/**/*.class" />
  </target>

  <target name="dist" depends="jar">
    <copy todir="${build.war.home}">
      <fileset dir="${web.home}"/>
    </copy>

    <copy todir="${build.war.home}/WEB-INF/lib">
      <fileset dir="${build.jar.home}" includes="*.jar"/>
    </copy>

    <mkdir dir="${dist.home}"/>
    <jar jarfile="${dist.home}/${app.name}.war" basedir="${build.war.home}"/>
  </target>

  <target name="jsp" depends="dist">
    <delete dir="${basedir}/testjsp"/>
    <java classname="org.apache.jasper.JspC" fork="true">
      <arg line="-v -d ${basedir}/testjsp -p com.objectmentor.library.jsp -mapped -compile -webapp ${build.war.home}"/>
      <arg line="WEB-INF/pages/patrons/books/loanRecords.jsp"/>
      <classpath>
        <fileset dir="${catalina.home}/common/lib">
          <include name="*.jar"/>
        </fileset>
        <fileset dir="${catalina.home}/server/lib">
          <include name="*.jar"/>
        </fileset>
        <fileset dir="${catalina.home}/bin">
          <include name="*.jar"/>
        </fileset>
        <fileset dir="${build.war.home}/WEB-INF/lib">
          <include name="*.jar"/>
        </fileset>
        <pathelement location="/Developer/Java/Ant/lib/ant.jar"/>
      </classpath>
    </java>
    <jar jarfile="${build.jar.home}/jsp.jar" basedir="${basedir}/testjsp" 
         includes="**/jsp/**/*.class" 
      />
  </target>
</project>

Of course you need all the standard files and directories beneath ${build.war.home} to make this work. If you are using custom tags in your JSPs, make sure you have all the appropriate TLD files in your tld directory.

Notice that the ANT file invokes the JspC command line, rather than using the JspC ant task that comes with TOMCAT. All the docs will tell you to use that ant task. However, I found that it doesn’t work well when you have custom tags. Maybe I’m just dumb, or maybe there is a bug in Jasper; but the only way I found to get Jasper to generate the right code was to invoke it from the command line and explicitly pass the JSP files in as command line argument!. If you depend on either the ant task or the command line to scan the whole web app for all the JSP files, it seems to generate the wrong code. (See: this blog)

Now that we have a JAVA file, let’s take a look at it. First, here’s the JSP file.

<%@ page import="com.objectmentor.library.utils.DateUtil" %>
<%@ page import="com.objectmentor.library.web.controller.patrons.LoanRecord" %>
<%@ page import="java.util.List" %>
<%
  List loanRecords = (List) request.getAttribute("loanRecords");
  if (loanRecords.size() > 0) {
%>
<table class="list" id="loanRecords">
  <tr>
    <th>ID</th>
    <th>Title</th>
    <th>Due date</th>
    <th>Fine</th>
  </tr>
  <%
    for (int i = 0; i < loanRecords.size(); i++) {
      LoanRecord loanRecord = (LoanRecord) loanRecords.get(i);
  %>
  <tr class="<%=i%2==0?"even":"odd"%>">
    <td><%=loanRecord.id%>
    </td>
    <td><%=loanRecord.title%>
    </td>
    <td><%=DateUtil.dateToString(loanRecord.dueDate)%>
    </td>
    <td><%=loanRecord.fine.toString()%>
    </td>
  </tr>
  <%
    }
  %>
</table>
<%
  }
%>

And here is the code that Jasper created from this file.

package com.objectmentor.library.jsp.WEB_002dINF.pages.patrons.books;

import javax.servlet.*;
import javax.servlet.http.*;
import javax.servlet.jsp.*;
import com.objectmentor.library.utils.DateUtil;
import com.objectmentor.library.web.controller.patrons.LoanRecord;
import java.util.List;

public final class loanRecords_jsp extends org.apache.jasper.runtime.HttpJspBase
    implements org.apache.jasper.runtime.JspSourceDependent {

  private static java.util.List _jspx_dependants;

  public Object getDependants() {
    return _jspx_dependants;
  }

  public void _jspService(HttpServletRequest request, HttpServletResponse response)
        throws java.io.IOException, ServletException {

    JspFactory _jspxFactory = null;
    PageContext pageContext = null;
    HttpSession session = null;
    ServletContext application = null;
    ServletConfig config = null;
    JspWriter out = null;
    Object page = this;
    JspWriter _jspx_out = null;
    PageContext _jspx_page_context = null;

    try {
      _jspxFactory = JspFactory.getDefaultFactory();
      response.setContentType("text/html");
      pageContext = _jspxFactory.getPageContext(this, request, response,
                  null, true, 8192, true);
      _jspx_page_context = pageContext;
      application = pageContext.getServletContext();
      config = pageContext.getServletConfig();
      session = pageContext.getSession();
      out = pageContext.getOut();
      _jspx_out = out;

      out.write('\n');
      out.write('\n');
      out.write('\n');

  List loanRecords = (List) request.getAttribute("loanRecords");
  if (loanRecords.size() > 0) {

      out.write("\n");
      out.write("<table class=\"list\" id=\"loanRecords\">\n");
      out.write("  <tr>\n");
      out.write("    <th>ID</th>\n");
      out.write("    <th>Title</th>\n");
      out.write("    <th>Due date</th>\n");
      out.write("    <th>Fine</th>\n");
      out.write("  </tr>\n");
      out.write("  ");

    for (int i = 0; i < loanRecords.size(); i++) {
      LoanRecord loanRecord = (LoanRecord) loanRecords.get(i);

      out.write("\n");
      out.write("  <tr class=\"");
      out.print(i%2==0?"even":"odd");
      out.write("\">\n");
      out.write("    <td>");
      out.print(loanRecord.id);
      out.write("\n");
      out.write("    </td>\n");
      out.write("    <td>");
      out.print(loanRecord.title);
      out.write("\n");
      out.write("    </td>\n");
      out.write("    <td>");
      out.print(DateUtil.dateToString(loanRecord.dueDate));
      out.write("\n");
      out.write("    </td>\n");
      out.write("    <td>");
      out.print(loanRecord.fine.toString());
      out.write("\n");
      out.write("    </td>\n");
      out.write("  </tr>\n");
      out.write("  ");

    }

      out.write("\n");
      out.write("</table>\n");

  }

    } catch (Throwable t) {
      if (!(t instanceof SkipPageException)){
        out = _jspx_out;
        if (out != null && out.getBufferSize() != 0)
          out.clearBuffer();
        if (_jspx_page_context != null) _jspx_page_context.handlePageException(t);
      }
    } finally {
      if (_jspxFactory != null) _jspxFactory.releasePageContext(_jspx_page_context);
    }
  }
}

Rant About Final.

A quick perusal of this code tells us that in order to use an instance of this servlet we need an HttpServletRequest instance, and an HttpServletResponse instance.

Looking a little closer we find that the servlet writes all the HTML to an instance of JspWriter that it gets from something called a PageContext. So, if we could create a mocked up version of JspWriter that saves all this HTML, and a mocked up version of PageContext that delivers the mocked JspWriter, then we could access the written HTML from our tests.

Fortunately the TOMCAT designers decoupled the creation of the JspWriter into a factory named JspFactory. That factory can be overridden! This means that we can get our mocked up JspWriter into the servlet without modifying the servlet. All we need is something like the following code:

  class MockJspFactory extends JspFactory {
    public PageContext getPageContext(Servlet servlet, ServletRequest servletRequest, ServletResponse servletResponse, String string, boolean b, int i, boolean b1) {
      return new MockPageContext(new MockJspWriter());
    }

    public void releasePageContext(PageContext pageContext) {
    }

    public JspEngineInfo getEngineInfo() {
      return null;
    }
  }

Now we need the mocked JspWriter. For the purposes of this demonstration, I used the following:

MockJspWriter

package com.objectmentor.library.web.framework.mocks;

import javax.servlet.jsp.JspWriter;
import java.io.IOException;

public class MockJspWriter extends JspWriter {

  private StringBuffer submittedContent;

  public MockJspWriter(int bufferSize, boolean autoFlush) {
    super(bufferSize, autoFlush);
    submittedContent = new StringBuffer();
  }

  public String getContent() {
    return submittedContent.toString();
  }

  public void print(String arg0) throws IOException {
    submittedContent.append(arg0);
  }

  public void write(char[] arg0, int arg1, int arg2) throws IOException {
    for (int i=0; i<arg2; i++)
      submittedContent.append(String.valueOf(arg0[arg1++]));
  }

  public void write(String content) throws IOException {
    submittedContent.append(content);
  }

  // lots of uninteresting methods elided.  I just gave them
  // degenerate implementations.  (e.g. {})
}

Don’t be concerned about the comment regarding all the unimplemented methods I elided. I’ve taken the attitude that I will only implement those methods that I need to get my tests to work. The others I leave with degenerate implementations.

My IDE was very helpful in creating these mocks. It automatically builds method prototypes and degenerate implementations for all the methods of an interface or abstract class that need implementing.

The MockPageContext, MockHttpServletRequest and MockHttServletResponse classes were created in a similar way.

MockPageContext

package com.objectmentor.library.web.framework.mocks;

import javax.servlet.*;
import javax.servlet.http.*;
import javax.servlet.jsp.*;
import java.io.IOException;
import java.util.Enumeration;

public class MockPageContext extends PageContext {

  private final JspWriter out;
  private HttpServletRequest request;

  public MockPageContext(JspWriter out) {
    this.out = out;
    request = new MockHttpServletRequest();
  }

  public JspWriter getOut() {
    return out;
  }

  public ServletRequest getRequest() {
    return request;
  }
  // lots of degenerate functions elided.
}

MockHttpServletRequest

package com.objectmentor.library.web.framework.mocks;

import javax.servlet.*;
import javax.servlet.http.*;
import java.io.*;
import java.security.Principal;
import java.util.*;

public class MockHttpServletRequest implements HttpServletRequest {

  private String method;
  private String contextPath;
  private String requestURI;
  private HttpSession session = new MockHttpSession();
  private Map parameters = new HashMap();
  private Map attributes = new HashMap();

  public MockHttpServletRequest(String method, String contextPath,
                                String requestURI) {
    super();
    this.method = method;
    this.contextPath = contextPath;
    this.requestURI = requestURI;
  }

  public MockHttpServletRequest() {
    this("GET");
  }

  public MockHttpServletRequest(String method) {
    this(method, "/Library", "/Library/foo/bar.jsp");
  }

  public String getContextPath() {
    return contextPath;
  }

  public String getMethod() {
    return method;
  }

  public String getRequestURI() {
    return requestURI;
  }

  public String getServletPath() {
    return requestURI.substring(getContextPath().length());
  }

  public HttpSession getSession() {
    return session;
  }

  public HttpSession getSession(boolean arg0) {
    return session;
  }

  public Object getAttribute(String arg0) {
    return attributes.get(arg0);
  }

  public String getParameter(String arg0) {
    return (String) parameters.get(arg0);
  }

  public Map getParameterMap() {
    return parameters;
  }

  public Enumeration getParameterNames() {
    return null;
  }

  public void setSession(HttpSession session) {
    this.session = session;
  }

  public void setParameter(String s, String s1) {
    parameters.put(s, s1);
  }

  public void setAttribute(String name, Object value) {
    attributes.put(name, value);
  }

  // Lots of degenerate methods elided.
}

MockHttpServletResponse

package com.objectmentor.library.web.framework.mocks;

import javax.servlet.ServletOutputStream;
import javax.servlet.http.*;
import java.io.*;
import java.util.Locale;

public class MockHttpServletResponse implements HttpServletResponse {
  // all functions are implemented to be degenerate.
}

Given these mock objects, I can now create an instance of my loanRecords_jsp servlet and start calling methods on it! Indeed, my first test case looked something like this:

  public void testSimpleTest() throws Exception {
    MockJspWriter jspWriter = new MockJspWriter();
    MockPageContext pageContext = new MockPageContext(jspWriter);
    JspFactory.setDefaultFactory(new MockJspFactory(pageContext));
    HttpJspBase jspPage = new loanRecords_jsp();
    HttpServletRequest request = new MockHttpServletRequest();
    HttpServletResponse response = new MockHttpServletResponse();

    jspPage._jspInit();
    jspPage._jspService(request, response);

    assertEquals("", jspWriter.getContent());
  }

The test fails, as expected, because the content is a bit more than blank – but not much more. If you look carefully at the JSP file you’ll see that it calls request.getAttribute("loanRecords") and expects a List back. But since our test did not create such an attribute, the code throws an exception that is silently caught.

To get real HTML out of this servlet, we need to load up the attributes. Then we can use HtmlUnit to parse that HTML and write tests against it.

HtmlUnit is very simple to use, especially for testing generated web pages like this. There are a few gotcha’s that I described here

Here is the final test that loads the attributes, inspects the HTML with htmlUnit, and passes appropriately:

package com.objectmentor.library.jspTest.books.patrons.books;

import com.gargoylesoftware.htmlunit.*;
import com.gargoylesoftware.htmlunit.html.*;
import com.objectmentor.library.jsp.WEB_002dINF.pages.patrons.books.loanRecords_jsp;
import com.objectmentor.library.utils.*;
import com.objectmentor.library.web.controller.patrons.LoanRecord;
import com.objectmentor.library.web.framework.mocks.*;
import junit.framework.TestCase;
import org.apache.jasper.runtime.HttpJspBase;

import javax.servlet.*;
import javax.servlet.http.*;
import javax.servlet.jsp.*;
import java.util.*;

public class LoanRecordsJspTest extends TestCase {
  private MockPageContext pageContext;
  private MockJspWriter jspWriter;
  private JspFactory mockFactory;
  private MockHttpServletResponse response;
  private MockHttpServletRequest request;
  private WebClient webClient;
  private TopLevelWindow dummyWindow;

  protected void setUp() throws Exception {
    jspWriter = new MockJspWriter();
    pageContext = new MockPageContext(jspWriter);
    mockFactory = new MockJspFactory(pageContext);

    JspFactory.setDefaultFactory(mockFactory);
    response = new MockHttpServletResponse();
    request = new MockHttpServletRequest();
    webClient = new WebClient();
    webClient.setJavaScriptEnabled(false);
    dummyWindow = new TopLevelWindow("", webClient);
  }

  public void testLoanRecordsPageGeneratesAppropriateTableRows() throws Exception {
    HttpJspBase jspPage = new loanRecords_jsp();
    jspPage._jspInit();

    List<LoanRecord> loanRecords = new ArrayList<LoanRecord>();
    addLoanRecord(loanRecords,
                  "99",
                  "Empire",
                  DateUtil.dateFromString("2/11/2007"),
                  new Money(4200));
    addLoanRecord(loanRecords,
                  "98",
                  "Orbitsville",
                  DateUtil.dateFromString("2/12/2007"),
                  new Money(5200));

    request.setAttribute("loanRecords", loanRecords);

    jspPage._jspService(request, response);

    StringWebResponse stringWebResponse = new StringWebResponse(jspWriter.getContent());
    HtmlPage page = HTMLParser.parse(stringWebResponse, dummyWindow);
    HtmlElement html = page.getDocumentElement();

    HtmlTable table = (HtmlTable) html.getHtmlElementById("loanRecords");
    List<HtmlTableRow> rows = table.getHtmlElementsByTagName("tr");
    assertEquals(3, rows.size());

    assertEquals("even", classOfElement(rows.get(1)));
    assertEquals("odd", classOfElement(rows.get(2)));

    List<HtmlTableDataCell> firstRowCells = rows.get(1).getCells();
    assertEquals(4, firstRowCells.size());

    List<HtmlTableDataCell> secondRowCells = rows.get(2).getCells();
    assertEquals(4, secondRowCells.size());

    assertLoanRecordRowEquals("99", "Empire", "02/11/2007", "$42.00", firstRowCells);
    assertLoanRecordRowEquals("98", "Orbitsville", "02/12/2007", "$52.00", secondRowCells);
  }

  private String classOfElement(HtmlTableRow firstDataRow) {return firstDataRow.getAttributeValue("class");}

  private void assertLoanRecordRowEquals(String id, String title, String dueDate, String fine, List<HtmlTableDataCell> rowCells) {
    assertEquals(id, rowCells.get(0).asText());
    assertEquals(title, rowCells.get(1).asText());
    assertEquals(dueDate, rowCells.get(2).asText());
    assertEquals(fine, rowCells.get(3).asText());
  }

  private void addLoanRecord(List<LoanRecord> loanRecords, String id, String title, Date dueDate, Money fine) {
    LoanRecord loanRecord = new LoanRecord();
    loanRecord.id = id;
    loanRecord.title = title;
    loanRecord.dueDate = dueDate;
    loanRecord.fine = fine;

    loanRecords.add(loanRecord);
  }

  private class MockJspFactory extends JspFactory {
    private PageContext pageContext;
    public MockJspFactory(PageContext pageContext) {
      this.pageContext = pageContext;
    }

    public PageContext getPageContext(Servlet servlet, ServletRequest servletRequest, ServletResponse servletResponse, String string, boolean b, int i, boolean b1) {
      return pageContext;
    }

    public void releasePageContext(PageContext pageContext) {
    }

    public JspEngineInfo getEngineInfo() {
      return null;
    }
  }
}

This test makes sure that the generated HTML has the appropriate content contained within the rows of a table. The test does expect to see a table, and expects the rows to appear in the appropriate order. It also ensures that the rows will have alternating styles. Other than that, all issues of form and syntax are ignored.

Conclusion

The technique described here can be used to test virtually any static web page, or portion thereof outside of a container, and without a webserver running. It is relatively simple to set up; and then very easy to extend. With it, you can spin around the edit/compile/test loop very quickly, and can easily follow the rules of Test Driven Development.

Testing GUIs Part I: Ruby on Rails

In the world of web development, no community has solved the problem of GUI testing better than the Ruby on Rails community. When you work on a rails project, testing the GUI is simply de-rigeur. The rails framework provides all the necessary tools and access points for testing all aspects of the application, including the generation of HTML and the structure of the resulting web pages.

Web pages in rails are specified by .rhtml files that contain a mixture of HTML and ruby code similar to the way Java and HTML are mixed in .jsp files. The difference is that .rhtml files are translated at runtime rather than being compiled into servlets the way .jsp pages are. This makes it very easy for the rails environment to generate the HTML for a web page outside of the web container. Indeed, the web server does not need to be running.

This ease and portability of generating HTML means that the rails test framework merely needs to set up the variables needed by the ruby scriptlets within the .rhtml files, generate the HTML, and then parse that HTML into a form that the tests can query.

A typical example.

The tests query the HTML using an xpath-like syntax coupled with a suite of very powerful assertion functions. The best way to understand this is to see it. So here is a simple file named: autocomplete_teacher.rhtml.

<ul class="autocomplete_list"> 
<% @autocompleted_teachers.each do |t| %> 
<li class="autocomplete_item"><%= "#{create_name_adornment(t)} #{t.last_name}, #{t.first_name}"%></li> 
<% end %> 
</ul> 

You don’t have to be a ruby programmer to understand this. All it is doing is building an HTML list. The Ruby scriptlet between <% and %> tokens simple loops for each teacher creating an <li> tag from an “adornment”, and the first and last name. (The adornment happens to be the database id of the teacher in parentheses.) A simple test for this .rhtml file is:

  def test_autocomplete_teacher_finds_one_in_first_name
    post :autocomplete_teacher, :request=>{:teacher=>"B"}
    assert_template "autocomplete_teacher" 
    assert_response :success
    assert_select "ul.autocomplete_list" do
      assert_select "li.autocomplete_item", :count => 1
      assert_select "li", "(1) Martin, Bob" 
    end
  end

• The post statement simply invokes the controller that would normally be invoked by a POST url of the form: POST /teachers/autocomplete_teacher with the teacher parameter set to "B".
• The first assertion makes sure that the controller rendered the autocomplete_teacher.rhtml template.
• The next makes sure that the controller returned success.
• the third is a compound assertion that starts by finding the <ul> tag with a class="autocomplete_list" attribute. (Notice the use of css syntax.)
  • Within this tag there should be an <li> tag with a class="autocomplete_item" attribute,
  • and containing the text (1) Martin, Bob.

It should not come as any surprise that this test runs in a test environment in which the database has been pre-loaded with very specific data. For example, this test database always has “Bob Martin” being the first row (id=1) in the Teacher table.

The assert_select function is very powerful, and allows you to query large and complex HTML documents with surgical precision. Although this example give you just a glimpse of that power, you should be able to see that the rails testing scheme allows you to test that all the scriptlets in an .rhtml file are behaving correctly, and are correctly extracting data from the variables set by the controller.

An example using RSpec and Behavior Driven Design.

What follows is a more significant rails example that uses an alternate testing syntax known as Behavior Driven Design (BDD). The tool that accepts this syntax is called RSpec.

Imagine that we have a page that records telephone messages taken from teachers at different schools. Part of that page might have an .rhtml syntax that looks like this:

<h1>Message List</h1>

<table id="list">
  <tr class="list_header_row">
    <th class="list_header">Time</th>
    <th class="list_header">Caller</th>
    <th class="list_header">School</th>
    <th class="list_header">IEP</th>
  </tr>
<%time_chooser = TimeChooser.new%> 
<% for message in @messages %>
  <%cell_class = cycle("list_content_even", "list_content_odd")%>
  <tr id="list_content_row">
    <td id="time"   class="<%=cell_class%>"><%=h(time_chooser.format_time(message.time)) %></td>
    <td id="caller" class="<%=cell_class%>"><%=h person_name(message.caller) %></td>
    <td id="school" class="<%=cell_class%>"><%=h message.school.name %></td>
    <td id="iep"    class="<%=cell_class%>"><%=h (message.iep ? "X" : "") %></td>
  </tr>
<% end %>
</table>

Clearly each message has a time, caller, school, and some kind of boolean field named “IEP”. We can test this .rhtml file with the following RSpec specification:

context "Given a request to render message/list with one message the page" do
  setup do
    m = mock "message" 
    caller = mock "person",:null_object=>true
    school = mock "school" 

    m.should_receive(:school).and_return(school)
    m.should_receive(:time).and_return(Time.parse("1/1/06"))
    m.should_receive(:caller).any_number_of_times.and_return(caller)
    m.should_receive(:iep).and_return(true)

    caller.should_receive(:first_name).and_return("Bob")
    caller.should_receive(:last_name).and_return("Martin")

    school.should_receive(:name).and_return("Jefferson")
    assigns[:messages]=[m]

    assigns[:message_pages] = mock "message_pages", :null_object=>true
    render 'message/list'
  end

  specify "should show the time" do
    response.should_have_tag :td, :content=>"12:00 AM 1/1", :attributes=>{:id=>"time"}
  end

  specify "should show caller first and last name" do
    response.should_have_tag :td, :content=>"Bob Martin", :attributes=>{:id=>"caller"}
  end

  specify "should show school name" do
    response.should_have_tag :td, :content=>"Jefferson", :attributes=>{:id=>"school"}
  end

  specify "should show the IEP field" do
    response.should_have_tag :td, :content=>"X",:attributes=>{:id=>"iep"}
  end
end

I’m not going to explain the setup function containing all that mock stuff you see at the start. Let me just say that the mocking facilities of RSpec are both powerful and convenient. Actually you shouldn’t have too much trouble understanding the setup if you try; but understanding it is not essential for this example. The interesting testing is in the specify blocks.

You shouldn’t have too much trouble reading the specify blocks. You can understand all of them if you understand the first. Here is what it does:

• The first spec ensures that <td id="time">12:00 AM 1/1</td> exists in the HTML document. This is not a string compare. Rather it is a semantic equivalence. Whitespace, and other attributes and complications are ignored. This spec will pass as long as there is a td tag with the appropriate id and contents.

HTML Testing Discipline and Strategy

One of the reasons that GUI testing has been so problematic in the .jsp world is that the java scriptlets in those files often reach out into the overall application domain and touch code that ties them to the web container and the application server. For example, if you make a call from a .jsp page to a database gateway, or an entity bean, or some other structure that is tied to the database; then in order to test the .jsp you have to have the full enabling context running. Rails gets away with this because the enabling context is lightweight, portable, and disconnected from the web container, and the live database. Even so, rails applications are not always as decoupled as they should be.

In Rails, Java, or any other web context, the discipline should be to make sure that none of the scriptlets in the .jsp, .rhtml, etc. files know anything at all about the rest of the application. Rather, the controller code should load up data into simple objects and pass them to the scriptlets (typically in the attributes field of the HttpServletRequest object or its equivalent). The scriptlets can fiddle with the format of this data (e.g. data formats, money formats, etc.) but should not do any calculation, querying, or other business rule or database processing. Nor should the scriptlets navigate through the model objects or entities. Rather the controller should do all the navigating, gathering, and calculating and present the data to the scriptlets in a nice little package.

If you follow this simple design discipline, then your web pages can be generated completely outside of the web environment, and your tests can parse and inspect the html in a simple and friendly environment.

Conclusion

I’ll have more to say about RSpec in a future blog. BDD is an exciting twist on the syntax of testing, that has an effect far greater than the simple syntax shift would imply.

I hope this article has convinced you that the rails community has solved the problem of testing HTML generation. This solution can be easily extrapolated back to Java and .NET as future blogs in this series will show.

Clearly the problem of testing Javascript, and the ever more complex issues of Web2.0 and GTK are not addressed by this scheme. However, there are solutions for Javascript that we will investigate in future blogs in this series.

Finally, this technique does not test the integration and workflow of a whole application. Again, those are topics for later blogs.

I hope this kickoff blog has been informative. If you have a comment, question, or even a rant, please don’t hesitate to add a comment to this blog.