Object Mentor Blog: Another Kata: Expression Tokenizer

Another Kata: Expression Tokenizer

Brett Schuchert — Mon, 29 Jun 2009 01:49:00 -0500

I continue to work on some examples for upcoming kata’s at the OkC CoCo. The next problem (previously mentioned here) is an implementation of the Shunting Yard Algorithm.

This past week I used that with a group in North Carolina and I discovered that it was much easier to tokenize the expression as I worked my way through it rather than attempt to make it easy to split into tokens up front.

I worked through just this smaller problem of converting an expression into tokens using the iterator pattern a few times and eventually got what I thought was a pretty good solution. I was wrong, I got it even tighter by making more appropriate use of the regular expression libraries in Java. (This is an example of where my regex mental model – based primarily on grep, vi and AWK – was inadequate when taking into consideration typical implementations of regex libraries with classes like Pattern and Matcher.)

Anyway, here’s a writeup for that Kata: http://schuchert.wikispaces.com/Katas.ExpressionTokenizer.

My final version turned into something crazy-short. I did not anticipate the final version. Part of it, as mentioned above, had to do with how I previously envisioned using regular expressions. Part of it might have been that it just took a few times for the patterns naturally occurring to become evident to me. In any case, working through this problem three times resulted in the following revelations:

Initially I though it would be hard, it was not and in fact much less work than what I was doing in the original problem. And it took a lot less time than I expected.
The second time around I was looking to duplicate the experience, which I did. But I saw quite a bit of improvement.
The final version was really an a-ah for me. Initially it was just like the second version. Then, once I had the tests in the kata written, I started messing around, looking for additional improvements (and heavily applying the DRY principle). I found them, exploited commonality and finally I got to the point where there was one thing I did not like, and fixing that led to a huge improvement, which led to a few more significant improvements.

Ultimately, I’ll have several useful take-aways from working on the problem several times.

If you’d like to see the “latest final version” of this, ping me. I’ll consider posting it to the blog, but I’d rather not give anything away without a specific request.

"Another Kata: Expression Tokenizer" by Judaica store

Sat, 24 Jul 2010 22:28:10 -0500

The best blog

"Another Kata: Expression Tokenizer" by Aluminum

Tue, 15 Jun 2010 01:37:08 -0500

Great blog thanks.

"Another Kata: Expression Tokenizer" by http://www.panelal.co.il

Tue, 15 Jun 2010 01:36:27 -0500

Great blog thanks.

"Another Kata: Expression Tokenizer" by fff

Thu, 03 Jun 2010 07:35:53 -0500

asd

"Another Kata: Expression Tokenizer" by Eli

Wed, 07 Apr 2010 12:17:42 -0500

<a href=”http://www.clickon.co.il/he-il/about/what-is-affiliate-network.htm //”>??? ??? ???? ?? ?? ?? ????

"Another Kata: Expression Tokenizer" by FLV extractor

Wed, 07 Apr 2010 03:33:34 -0500

so it is

"Another Kata: Expression Tokenizer" by Eli

Sun, 28 Mar 2010 10:34:41 -0500

? ?? ??? ? ? ?? ???? ??? ?? ??

"Another Kata: Expression Tokenizer" by Itay

Sun, 28 Feb 2010 10:00:52 -0600

? ??? ?? ?? ?? ??? ???

"Another Kata: Expression Tokenizer" by itamar

Sun, 28 Feb 2010 09:59:03 -0600

? ?? ??? ??? ?? ?? ??,???? ?? ??? ???? ?? ? ?? ?? ???

"Another Kata: Expression Tokenizer" by ?????????

Tue, 01 Dec 2009 02:29:56 -0600

??? ?? ? ?? ????? ?? ?? ?? ? ???? ?? ??? ? ??? ??? ? ??? ??

"Another Kata: Expression Tokenizer" by daniel

Tue, 01 Dec 2009 02:29:32 -0600

??? ?? ? ?? ????? ?? ?? ?? ? ???? ?? ??? ? ??? ??? ? ??? ??

"Another Kata: Expression Tokenizer" by ??????? ???????

Thu, 19 Nov 2009 09:52:38 -0600

?? ?? ?? ??? ?????

"Another Kata: Expression Tokenizer" by drums

Wed, 28 Oct 2009 03:33:56 -0500

ldf

"Another Kata: Expression Tokenizer" by ???? ???????

Wed, 12 Aug 2009 04:48:54 -0500

?? ?????

?? ?? – ??? ??? ???? ? – ?? ?????. ?? ??? ??????

?? ??? ??? ???? ?? 10 ?? 80 ???.

??? ?? ???? ???? ?? ?? ????.

?? ??? ?? ??? ???? ?: ??? – ??? ???? – ? ?? ????? .

?? ??? ? ?? ?? ??? ??? ?? ?? ?? ? ?? ?? ????. ??? ??? ?? ???? ??? ??? ?? ??? , (? ? ??? ?? ??? ?? ???? ) , ? ?? ?? ??? ? ?? ? ?? ???? ?? ??? ? ???? ?? ??? ??? , ???? , ??? ?? ???, ??? ??, ? ??? ?? ?? ?? ??? ’???? ’.

"Another Kata: Expression Tokenizer" by ???? ?????

Wed, 12 Aug 2009 04:46:45 -0500

??? ??? ?? ?? ??? ??? ??? ??? – ?? ??? ? ? ??? ?? ?? ??, ?? ?? ??? ? , ? ?? ??? ?? ?? ??? ???? ??? ? ?? ??.

"Another Kata: Expression Tokenizer" by ??????

Wed, 12 Aug 2009 04:45:29 -0500

? ??? ?? ???? ?? ?? ??? (??? ?? ??...) ???, ???, ???? ???- ?? ?? ?? ??. ?? ? ?? ???: ?? ??? ? ??? ???? ?? ???, ??? ? ???? ??? ? ??? ???, ?????? ????, ??? ?? ???, ???, ??? ??’?? ?? ??? ??-? ?? ?? ?? ??? ???? ?? ?? ??? ??, ?? ???? ??? ?? ??? ?? ??? ??, ???, ?????.

"Another Kata: Expression Tokenizer" by ????? ????????

Thu, 06 Aug 2009 00:37:46 -0500

?? ??? ?? ????? ???? . ??? ???? ?? ?? ? ?? ?? ? ?? ? ?? ? ?? . ?? ?? ????? ?? ???? ?? ??? ?? ? ?? ?? ?? ?? ??? ??? ?? ??? ?? ?? ?? ????? ???? ??? ???? ?? ??? ??? ??? ?? ?? ??? ?? ?? ??? . ?? ????? ? ?? ??? ? ??? ???? ??? ?? ??? ??? ?? ??? ???? ?? ????? . ?? ???? ????? ? ??? ?? ?? ? ?? ?? ?? ?? ?? ??? ?? ???? ??? ??? ??? ?? ?? ?? ????? ?? ?? ?? ?? ??? ? ??? ? ??? ?? ? ?? . ???? ????? ? ??? ??? ?? ?? ? ??? ??? ???? ??? ??? ??? ??? ??? ??? ???? ?? ?? 1500 ?? 2000 ???. ??. ? ?? ??? ???? ?? ?? ??? ?? ?? ?? ?? ?? ? ?? ?? ?? ???? ????? ?? ????? ? ?? ?? ? ??? ? ??? ??? ?? ??? ???? ????? ? ?? ?? ?? ??? ??? ?? ??? ???? ?? ??? ?? . ? ? ? ??? ??? ?? ?? ??? ??? ??? ?? ?? ? ?? ???? .

"Another Kata: Expression Tokenizer" by ????? ?????

Thu, 06 Aug 2009 00:37:14 -0500

? ?? ? ???? ??? ??. ??? ? ?? ???? ?? ??? ??? ? ?? ? ?? ?? ??. ??? ??? ???? ?? ??? ??? ???? ?? ?? ???? ???? . ?? ?? ??? ??? ?? ?? ???? ?? ?? ?? ??? ???? ???. ??? ?? ?? ?? ? ? ?? ?? ?? ??, ?? ?? ?? ? ?? ??? ??? ??? ?? ? ???? ?? ?? ??. ? ? ??? ??? ??? ?? ?? ?? ?? ? 13 ?? ?? ??”? ? ??, ??? ?? ?? ??? ? ???? ??? , ?? ??? ??? ?? ??? ?? ??? ?? ?? ?? ?? ?? ?? ?? ??? ??? ?? ??. ???? ?? ??? ?? ? ?? ??? ??? ??? ?? ?? ??? ???? ???. ?? ?? ? ?? ?? ?? ??? ??? ?? ??? ??? ??? ?? ?? ?? ??? ??? ?? ??? ???? ?? ? ?? ??? ??? ? ??? ?? ?? ?? ?? ?? ??? ?? ?? ?? ?? ?? ??? ??.?? . ?? ?? ?? ?? ?? ??? ??? ?? ??? ???. ??? ??? ?? ?? ? ??? ?? ?? 2000 ?? ??? ? ? ?? ??? ???? ?? ? ?? ? ??? ?? 1200 ?? . ?? ?? ?? ?? ?? ?? ?? ???, ? ? ?? ??? ?? ??? ???? ?? ?? ??? ? ?? ? ??? ???? ????? ??? ???? ???? ??.

"Another Kata: Expression Tokenizer" by ??? ???????

Mon, 03 Aug 2009 05:06:24 -0500

?? ?? ?? ? ???...

"Another Kata: Expression Tokenizer" by ????? ??? ??????

Mon, 03 Aug 2009 05:04:15 -0500

??? ?? ?? ? ? ??? ??????

"Another Kata: Expression Tokenizer" by ????? ???? ?????

Mon, 03 Aug 2009 05:01:46 -0500

? ? ??? ?? ??? ?? ?? ??? ??? ????

"Another Kata: Expression Tokenizer" by ??????? ??????

Mon, 03 Aug 2009 04:57:24 -0500

??? ?? ? ???? ??, ??? ??? ???? ??? ???

"Another Kata: Expression Tokenizer" by ????? ?????

Mon, 03 Aug 2009 04:55:00 -0500

??? ??? ??? , ??? ? ??

"Another Kata: Expression Tokenizer" by http://www.xn--5dbfbum6aa1cpp.com/images/spacer.gif

Mon, 03 Aug 2009 04:45:11 -0500

?? ??? ??? ??? ?? ?? ? ? http://www.xn—5dbfbum6aa1cpp.com/images/spacer.gif

??? ??? http://www.xn—5dbfbum6aa1cpp.com/

"Another Kata: Expression Tokenizer" by http://www.xn--5dbfbum6aa1cpp.com/

Mon, 03 Aug 2009 04:40:17 -0500

???? ? ???? ?? ?? ?? ?? ???? ?? ??