Wormholes, FitNesse and the return of SetUp and TearDown links

Wormholes, FitNesse and the return of SetUp and TearDown links 38

Posted by Brett Schuchert Thu, 16 Apr 2009 18:33:00 GMT

Recently, I was working on adding back a feature to FitNesse that had been removed, links at the bottom of each page to add a setup or teardown page to the current page. After racking my brain and spending time in git with file histories, I had discovered a point in time where the feature was there and the next commit where it was gone. It was not obvious to me what had changed to break the feature until I talked with Bob Martin (much of this has to do with my lack of experience using git). He mentioned a change in the handling of importing header and footer pages (related to my problem) and sure enough, when I took a look in the debugger, I found out that the information I needed to reintroduce the feature had essentially be removed as a result of a bug fix.

This was, apparently, not a heavily used feature. In fact, I had not used it much until I recently started working on tutorials for FitNesse. And given the release criterion for FitNesse, removal of the feature did not break anything (no acceptance tests nor unit tests).

Anyway, the point at which the information was available that I needed and the point where I needed to use the information were may steps away from each other both in the call stack as well as the instance hierarchy (think composite pattern). I did not want to significantly change the method call hierarchy so I instead decided to hand-roll the wormhole pattern as it manifests itself in AspectJ (and not the wormhole anti-pattern).

For background on AspectJ and AOP in general, have a look at these self-study tutorials.

The wormhole pattern is well-documented in AspectJ In Action. It consists of two point cuts that combine two different, separated, parts of the system. It grabs information available at (in this case) the entry point to the system and makes that information available deeper in the system without passing the parameter directly. That’s where the name comes from, the pattern bridges information across to unconnected points via the wormhole. It is my favorite patter name (not favorite pattern, just name).

AspectJ happens to store certain runtime information in thread local storage and the wormhole pattern exploits this fact. Here’s another example that’s close to the wormhole pattern. This is actually a common technique. If you’ve read up on the recommendations on using Hibernate in a JSE environment either in “Hibernate in Action” or the more recent Javer Persistence with Hibernate, one recommendation is to pass around sessions and such in thread local variables. Under the covers, JEE containers do the same thing.

Even though this is a “common” technique, I’m not a huge fan of using thread-locals. 1. They are thread-specific global variables. 1. You better be sure there’s no threading between the two points. In this case, the threading has already happened. Once a FitNesse responder gets a hold of an HTTP request, the remaining processing is in a single thread.

On the other hand, if I did not use thread local storage, the change required to get information I needed would have either require changing one of the objects already in the parameters being passed (very ugly) or changing method signatures all over the place (somewhat less ugly but a stronger violation of LSP). So in this case, I think thread local variables are the least ugly of the available options.

(As a side note, that’s my definition of design: Select the solution that sucks the least.)

If you’re like me, you don’t use thread locals very often. I wanted to make sure that the thread local information would get properly cleaned up and I wanted to hide all of the magic in one place, so I created a simple class called ThreadLocalUtil. I used TDD to create the class and when I though I was done, I wanted to make sure that I had written the clear() method correctly. I knew that for a single thread the clear() method worked as expected, but I wanted to make sure it did not affect other threads.

So my problem was I needed 2 threads and I wanted a particular ordering of events:

T1: Store value in thread-local storage.
T2: Clear its local storage.
T1: Read its local storage, value stored should still be available.

This really isn’t a hard test to write other than the ordering of events. To make that work, I used latches and I had the threads manually single each other. Here’s the test:

package fitnesse.threadlocal;

import static org.junit.Assert.assertEquals;
import static org.junit.Assert.assertNull;

import java.util.concurrent.CountDownLatch;

import org.junit.After;
import org.junit.Before;
import org.junit.Test;

public class ThreadLocalUtilTest {
  private String valueFound;

  // snip - several other tests removed to focus on this test

  CountDownLatch t1Latch = new CountDownLatch(1);
  CountDownLatch t2Latch = new CountDownLatch(1);

  class T1 implements Runnable {
    public void run() {
      try {
        ThreadLocalUtil.setValue("t1", "value");
        t2Latch.countDown();
        Thread.yield();
        t1Latch.await();
        valueFound = ThreadLocalUtil.getValue("t1");
      } catch (InterruptedException e) {
        e.printStackTrace();
      }
    }
  }

  class T2 implements Runnable {
    public void run() {
      try {
        t2Latch.await();
        ThreadLocalUtil.clear();
        t1Latch.countDown();
      } catch (InterruptedException e) {
        e.printStackTrace();
      }
    }
  }

  @Test
  public void assertThatClearInOneThreadDoesNotMessUpAnotherThread()
      throws InterruptedException {
    Thread t1 = new Thread(new T1());
    Thread t2 = new Thread(new T2());
    t1.start();
    t2.start();
    t1.join();
    t2.join();
    assertEquals("value", valueFound);
  }
}

This example illustrates using the java.util.concurrent.CountDownLatch class to signal between threads.

Main Test Method

Creates two threads and start them.
Wait for both threads to complete.

T1.run(), T2.run()

If T2 starts running before T1, no problem it waits on the countdown latch.
When T1 starts, it stores a value in a thread local using the util class.
T1 then counts down, releasing T2.
T1 then yields, it is done until its countdown latch is signaled.
T1 waits for a signal.
T2 is released from its countdown latch.
T2 sends clear to the thread local util class (there’s a test that verifies clear() works as named in a single thread.
T2 then signals T1 to continue by calling its countdown latch.
T2 completes at some point later.
T1 starts running again, grabs the value out of thread local storage and puts it in the valueFound.
T1 completes.

Main Test Method

The completion of T1 and T2 make the join methods called in the original test method to return, whereupon the test can complete.
Verifies that the variable valueFound, set in T1.run, stores the expected result.

This kind of hand-rolled synchronization is certainly error prone. This is where having a second pair of eyes can really help. However, this seemed to verify that the clear method as written worked in multiple threads as expected.

If you’re interested in seeing the ThreadLocalUtil class or ThreadLocalUtilTest class, grab a copy of FitNesse from github.

Posted in Schuchert's Scattered Synapses
Tags blocking, multi, non, test, threading, Unit
Meta 38 comments, permalink, rss, atom

Comments

Leave a response

Markus Gärtner about 2 hours later:

Nice introduction to unit-testing concurrent classes. Though I’m sure that I have just understood 80% of it on the first read, I’ll pick your description for use at work. Enlightening.
Esko Luontola about 5 hours later:

Also I have found CountDownLatch to be useful for writing repeatable concurrent tests. Here is one example:

*ttp://github.com/orfjackal/dimdwarf/blob/0d1a0897821a4e0e557a1390a7252a8580d2ec95/dimdwarf-core/src/test/java/net/orfjackal/dimdwarf/scheduler/TaskThreadPoolSpec.java#L130-192

The negative side is that tests which require that much coordination between threads tend to be very long and complex. It would be much better to design the program so that none of the classes need concurrency control. (I’ll refactor my program’s architecture in the following version to be message passing based, so that hopefully the only concurrent class will be a BlockingQueue. That should make the code much simpler and easier to test.)

Another tip that has helped me to find concurrency bugs is the one from Clean Code – run with more threads than processors. I do that by starting 20, 50 or 100 test runs simultanously (depending on how thorough I want to be and how many GBs the machine has memory). I hold down Shift+F10 in IDEA for a few seconds and look at the “N processes running” text in the status bar. Looking at Windows Task Manager, after the memory and CPU usage has returned to normal, I go through all the test run windows and look for any failed tests and error messages in the log (normally the stdout/err is empty). If a test has deadlocked, I do a thread dump. Then I investigate the reason prudently until I find the reason for the error (even if it fails only one time in a million, it’s still a bug which needs to be fixed).
Brett L. Schuchert about 9 hours later:
Esko Luontola

Another tip that has helped me to find concurrency bugs is the one from Clean Code…
Glad you liked that.
I’ve found that you can greatly reduce the number of threads to maybe one more than the number of your cores if you:
- Run your tests through a few iterations.
- Use IBM ConTest to mix up the ordering during test execution.
I have one example where 20 threads, each looping 10,000 times cannot expose a known bug better than 10% of the times the tests run. I can reduce it to 2 threads and a count of about 20 and run the same test with ConTest and demonstrate the defect better than 90% of the time.
Esko Luontola about 15 hours later:

Thanks. I’ll try ConTest some day.
http://www.linkedin.com/in/przemyslawpokrywka 5 days later:

Hi,

Have you considered using technique described at http://misko.hevery.com/2009/04/15/managing-object-lifetimes/ ? It solves the “add a parameter to every method in a call chain” problem in a very clean way. You could forget about ThreadLocals then. I’m just beginning to study the method, so I can not tell if it would be the best option for you, although it looks very promising to me.

Przemek
Jeff 16 days later:

Using the wormhole pattern is almost always a terrible idea. Adopting a threadlocal for the purpose is at best a hack based on a possibly incorrect assumption that the source and sink of the wormhole are evaluated on the same thread. At best the wormhole implementation should be based in some higher-level mutable context that is known to be available to all participants. (eg. http context, user data, property list, etc.)

The fact you cannot implement a given feature without passing data around by underhanded means is usually a sign that the object model needs to be refactored somehow.
Jeff 16 days later:

Using the wormhole pattern is almost always a terrible idea. Adopting a threadlocal for the purpose is at best a hack based on a possibly incorrect assumption that the source and sink of the wormhole are evaluated on the same thread. At best the wormhole implementation should be based in some higher-level mutable context that is known to be available to all participants. (eg. http context, user data, property list, etc.)

The fact you cannot implement a given feature without passing data around by underhanded means is usually a sign that the object model needs to be refactored somehow.
Brett L. Schuchert 17 days later:

Jeff wrote:

Using the wormhole pattern is almost always a terrible idea.

I agree. Before I started writing unit tests, I passed the idea by Bob to make sure I wasn’t crazy.

I determined that passing around the correct info would touch quite a bit of code (ugly). The object model had no idea of “current test page”, which is what I needed, and introducing such an idea would touch a few key objects and the construction thereof. (Also ugly.)

I happen to have researched the threading model of FitNesse, so I was pretty sure it was safe (and in fact it was), but that assumes there will be no changes to the threading model in the future. As it is, there won’t be any essential changes to the thread execution model anytime soon. FitNesse follows a servlet model. The threading is before the Servlet ever executes. Everything behind it is in a single thread (thread per request). Even if the underling implementation were changed to use Java 1.5 threading support, it would not affect the code in question. So while there is a theoretical problem, in practice it’s stable for the foreseeable future.

So it seemed the least ugly of the alternatives. However, my spider senses were tingling because that patter is fragile at best.

So then the question becomes, does it make sense to change the FitNesse object model? Maybe, but that would require more concentration time that I generally have available!-(
Designer Bags about 1 year later:

Thank u for sharing! It really helpful to me about those information.
Pandora about 1 year later:

It is my favorite patter name (not favorite pattern, just name).
pandora about 1 year later:

An important feature about buying on the web is you’re able to see everything the actual provides, whereby a store it is usually challenging type even so his or her products to locate the right thing. Moms really like products which are unforgettable. Attempt to found your mother using a surprise that may be personal, as a possible example a diamond ring applying their brand engraved within it. Gold bands lead to excellent items given that they immortalize your currentbrand in precious metal. A new gold pendant and also a pendant creates a great gifts on your mommy, as well as products which can be most often ignored .
http://www.whiteiphone4transformer.com about 1 year later:

If I got a chance, i will prefer buying the iphone 4 white but not the iphone 4 Black. Who can tell me where is the white iphone 4 available? I would really want to take one.
moncler about 1 year later:

thanks for Moncler Jackets || Christian louboutin UK || Moncler coats || Christian louboutin shoes || Christian louboutin pumps your post!
iPad to Mac Transfer about 1 year later:

iPad to Mac Transfer can help you transfer music, movie, photo, ePub, PDF, Audiobook, Podcast and TV Show from ipad to mac freely.
iPhone SMS to Mac Backup about 1 year later:

Would you like to banckup iphone SMS to mac, macBook, macbookPro as .txt files? Now a software iphone SMS to Mac Backup can help you to realize it.
iPad ePub Transfer for Mac about 1 year later:

I really like this essay. Thank you for writing it so seriously. I want to recommend it for my friends strongly. iPad ePub Transfer for Mac is a Professional transfer that can transfer the ePub books onto your iPad on Mac and export the iPad ePub to Mac local folder.
ray ban about 1 year later:

Ray Ban also has been a brand respected by the great numbers of sunglass wearers worldwide. The brand is extremely efficient for superior quality and design. Ray Ban UK has long been very trustworthy for its responsibility in the world. Thedesigners of Ray Ban sunglasses take advantage of three different materials to make distinct designs- metal, plastic and titanium. Titanium frames by Ray Ban Sale became extremely famous due to their unique properties. These frames are hypo allergic, corrosion-resistant and nickel-free. These a href=”http://www.raybanonsale.uk.com/”>frames are versatile and best for any kind of environment and weather, whether it’s hot, humid, cold or dry.
Polo Ralph Lauren about 1 year later:

Podcast and TV Show from ipad to mac freely.
RolexReplica about 1 year later:

It is advisable to own custom made Rolex Replicas to put on about any situation, be it unconventional as well as official. You might have reproduction watches, shopping watches several additional products.
Sheen Crow about 1 year later:

This is an awesome article. Thanks for this. roofing englewood
mac cosmetics over 2 years later:

Thanks for sharing~ I’ve been obsessed with Hikaru no Go doujinshi recently and I came across your site. It was wonderful >_<
chanel handbags for sale over 2 years later:

sobbing. how long has it been since i read a hikago doujin? one of dragonfly’s scanlations too.
christian louboutin sales over 2 years later:

Thank you! _
okey oyunu oyna over 2 years later:

thanks internette görüntülü olarak okey oyunu oyna, gerçek kisilerle tanis, turnuva heyecanini yasa.
christian louboutin over 2 years later:

good produces!
shoes christian louboutin over 2 years later:

Fuck Sam Goody, Fuck FYE, PEace the FUCK OUT
christian louboutin over 2 years later:

http://www.us-christianlouboutin.com/
Authentic Nike Jordan Shoes over 2 years later:

e, very Jordan 6 rings sorry. City Board of Education Jordan V learned Air Jordan 9 that the policy requiring students to military training is only popular Jordan V in Jordan Spizike high school, junior high school stage not required. There are some schools …
beats by dre store over 2 years later:

y friends strongly. iPad ePub Transfer for Mac is a Professional transfer that can transfer the ePub books onto your iPad on Mahigh quality headphones new design headphones
Tips For Bowling over 3 years later:

Yes, in general, I think there is too much club football on TV. Sepp Blatter
ghd over 3 years later:

Yes, in general, I think there is too much club football on TV. Sepp Blatter
<a href="http://www.wingsjshoes.com/adidasangrybirdsshoes-c34.html">Adidas Angry Birds</a> over 3 years later:

And finally found under
find cell number over 3 years later:

While most people could tell you the highest mountain in the world is Mount Everest and the longest river in the world is the Nile, not many people know much about the biggest lakes around the world.

hemorrhoid removal at home
iPhone contacts backup over 3 years later:

It is really a good example that most of the programmer should think about this. If we want to do much better for the code. I should understand it. right? You are very good at this and show us good coding idea.
mbtshoe over 3 years later:

Australia Beats By Dre Studio dr dre beats headphones beats studio beats pro beats solo hd pro headphones music Official store Monster Beats By Dre Pro
Diamond Cuts over 3 years later:

Be great if we could just stick to the subject.
beats headphons over 3 years later:

The monster beats headphons are semi-open in design

However, the headphone industry is greeted with far less unanimity. dr dre headphons targeted toward audiophiles are often considered weak or thin sounding by the general consumer. Despite a price tag worth more than 5 times the amount of the monster beats dre studio headphones they are used to, the consumer may not be inclined to agree with the more expensive headphone’s proclaimed superiority. In order to please both the consumer and audiophile demographics, headphone companies must either exhaust their R&D departments and often come up short, discover some magic voodoo potion, or simply give up! I know for certain that in the case of the Beats By Dr Dre Solo, V-Moda did not subscribe to the latter.The beats headphons headphones themselves are semi-open in design and allow sound to leak into the outside. For this reason, the Beats By Dr Dre may not be the ideal headphone choice for some listening environments.

Some may inquire as to why Beyerdynamic would choose to create an open-back headphone. The reason as specified by many manufacturers that design open-back New Style Beats By Dr Dre headphones is that it is easie.House of Marley has consistently surprised us with their uniquely designed monster beats dre studio headphones. The House of Marley Trenchtown Rock Destiny is their latest offering, a noise cancelling headphone which competes with many similarly-priced noise cancelling monster beats dre studio headphones and ‘rapper monster beats dre studio headphones’. Overall I felt that the Destiny is a good choice for a listener who likes great style, active noise cancellation and up-front forward bass.The noise-cancelling ability of these cheap beats by dre studio headphones is pretty good, however there may be better options at this price point if Active Noise Cancellation is more important to you than the aesthetics of the cans. I do however have two quibbles with regard to the way the feature is implemented. One is that in order to hear music, you must use the noise cancelling feature.

This means that your batteries better not run out, or you should be prepared with extra batteries (two triple A). While it is not uncommon for noise cancelling Beats By Dr Dre Pro to require the noise cancelling feature to be activated in order to hear music, several new models are coming out that will still output sound when batteries are dead, or noise cancelling is in passive mode. The second is the louder than average thump sound which one hears when switching the noise cancellation on while the monster beats dre studio headphones are on the head.Listening to Maroon Five’s “Moves Like Jagger” I was very impressed with the immersive quality this headphone had. If you want a headphone that makes you dance, these Destiny monster beats dre studio headphones may very well be your destiny.With regard to accessories, beats dre studio do not offer much. The cable itself is built into the headphone (Y-split style) and is not user replaceable. The cable itself is very sturdy, extra thick and may benefit from a Beats By Dr Dre Solo headphone extension cable if one is going to be more than 4 feet from their source.
dr dre headphons,beats headphons,New Style Beats By Dr Dre,cheap beats by dre studio,Beats By Dr Dre Pro,Beats By Dr Dre Solo
louboutin sales over 3 years later:

Wormholes, FitNesse and the return of SetUp and TearDown links 37 hoo,good article!!I like the post!128

Mentor	twitter id
Uncle Bob	unclebobmartin
Brett Schuchert	schuchert
Michael Feathers	mfeathers
Bob Koss	bob_koss

Wormholes, FitNesse and the return of SetUp and TearDown links 38

Comments

Blog Search

Follow us on twitter

Categories

Blogroll

Syndicate