Another Kata: Expression Tokenizer

Another Kata: Expression Tokenizer 76

Posted by Brett Schuchert Mon, 29 Jun 2009 06:49:00 GMT

I continue to work on some examples for upcoming kata’s at the OkC CoCo. The next problem (previously mentioned here) is an implementation of the Shunting Yard Algorithm.

This past week I used that with a group in North Carolina and I discovered that it was much easier to tokenize the expression as I worked my way through it rather than attempt to make it easy to split into tokens up front.

I worked through just this smaller problem of converting an expression into tokens using the iterator pattern a few times and eventually got what I thought was a pretty good solution. I was wrong, I got it even tighter by making more appropriate use of the regular expression libraries in Java. (This is an example of where my regex mental model – based primarily on grep, vi and AWK – was inadequate when taking into consideration typical implementations of regex libraries with classes like Pattern and Matcher.)

Anyway, here’s a writeup for that Kata: http://schuchert.wikispaces.com/Katas.ExpressionTokenizer.

My final version turned into something crazy-short. I did not anticipate the final version. Part of it, as mentioned above, had to do with how I previously envisioned using regular expressions. Part of it might have been that it just took a few times for the patterns naturally occurring to become evident to me. In any case, working through this problem three times resulted in the following revelations:

Initially I though it would be hard, it was not and in fact much less work than what I was doing in the original problem. And it took a lot less time than I expected.
The second time around I was looking to duplicate the experience, which I did. But I saw quite a bit of improvement.
The final version was really an a-ah for me. Initially it was just like the second version. Then, once I had the tests in the kata written, I started messing around, looking for additional improvements (and heavily applying the DRY principle). I found them, exploited commonality and finally I got to the point where there was one thing I did not like, and fixing that led to a huge improvement, which led to a few more significant improvements.

Ultimately, I’ll have several useful take-aways from working on the problem several times.

If you’d like to see the “latest final version” of this, ping me. I’ll consider posting it to the blog, but I’d rather not give anything away without a specific request.

Posted in Schuchert's Scattered Synapses
Tags kata, practice, regex
Meta 76 comments, permalink, rss, atom

Comments

Leave a response

Barak about 1 month later:

??

?? ???? ? ??? ??? ??? ??? ???? ??? ????, ? ??? ???? ?? ?? ??? ??? ???? ? ???, ? ?? ??? ?’. ? ?? ?? ?, ?? ??? ??? ????? ??? ??? ???. ??? ?? ???? ??? ???? -? ?? ?? ? ?? ?? ?? ?? ????.

?? ???? ??? ??? MBK ??, ?? ? ??? ?? T.U.V- ?? ?? ??.

?? ?? ?? ??? ?? ?? ?? ???, ?”? ??? ??? ???? ?? ???, ?? ?”? ?? ??? ??? (??), ?? ??? ?? ?? ?.
?????? about 1 month later:

-? ?? ?? ? ?? ?? ?? ?? ????.

?? ???? ??? ??? MBK ??, ?? ? ??? ?? T.U.V- ?? ?? ??.

?? ?? ?? ??? ?? ?? ?? ???, ?”? ??? ??? ???? ?? ???, ?? ?”? ?? ??? ??? (??), ?? ??? ?? ?? ?.
??????? about 1 month later:

?? ? ? ? ?? ?? ???, ? ? ?? ?? ??. ?? ? ?? ???? ?? ??? ? ???? ???? ??. ??? ?? ?? ?? ?, ? ?? ??? ? ?? ??? ?? ?? ???. ??? ?? ?? ? ?? ? ???? ?? ?”? ??. ?? ?? ??? ???, ??? ?? ???? ??? ???. ?? ? ?? ?? ???, ?? ??? ??? ?? ??? ?? ?? ?? ??? ??. ??? ?, ??? ??? ?? ?? ??? ?? ??.
http://www.xn--5dbfbum6aa1cpp.com/ about 1 month later:

???? ? ???? ?? ?? ?? ?? ???? ?? ??
http://www.xn--5dbfbum6aa1cpp.com/images/spacer.gif about 1 month later:

?? ??? ??? ??? ?? ?? ? ? http://www.xn—5dbfbum6aa1cpp.com/images/spacer.gif

??? ??? http://www.xn—5dbfbum6aa1cpp.com/
????? ????? about 1 month later:

??? ??? ??? , ??? ? ??
??????? ?????? about 1 month later:

??? ?? ? ???? ??, ??? ??? ???? ??? ???
????? ???? ????? about 1 month later:

? ? ??? ?? ??? ?? ?? ??? ??? ????
????? ??? ?????? about 1 month later:

??? ?? ?? ? ? ??? ??????
??? ??????? about 1 month later:

?? ?? ?? ? ???...
????? ????? about 1 month later:

? ?? ? ???? ??? ??. ??? ? ?? ???? ?? ??? ??? ? ?? ? ?? ?? ??. ??? ??? ???? ?? ??? ??? ???? ?? ?? ???? ???? . ?? ?? ??? ??? ?? ?? ???? ?? ?? ?? ??? ???? ???. ??? ?? ?? ?? ? ? ?? ?? ?? ??, ?? ?? ?? ? ?? ??? ??? ??? ?? ? ???? ?? ?? ??. ? ? ??? ??? ??? ?? ?? ?? ?? ? 13 ?? ?? ??”? ? ??, ??? ?? ?? ??? ? ???? ??? , ?? ??? ??? ?? ??? ?? ??? ?? ?? ?? ?? ?? ?? ?? ??? ??? ?? ??. ???? ?? ??? ?? ? ?? ??? ??? ??? ?? ?? ??? ???? ???. ?? ?? ? ?? ?? ?? ??? ??? ?? ??? ??? ??? ?? ?? ?? ??? ??? ?? ??? ???? ?? ? ?? ??? ??? ? ??? ?? ?? ?? ?? ?? ??? ?? ?? ?? ?? ?? ??? ??.?? . ?? ?? ?? ?? ?? ??? ??? ?? ??? ???. ??? ??? ?? ?? ? ??? ?? ?? 2000 ?? ??? ? ? ?? ??? ???? ?? ? ?? ? ??? ?? 1200 ?? . ?? ?? ?? ?? ?? ?? ?? ???, ? ? ?? ??? ?? ??? ???? ?? ?? ??? ? ?? ? ??? ???? ????? ??? ???? ???? ??.
????? ???????? about 1 month later:

?? ??? ?? ????? ???? . ??? ???? ?? ?? ? ?? ?? ? ?? ? ?? ? ?? . ?? ?? ????? ?? ???? ?? ??? ?? ? ?? ?? ?? ?? ??? ??? ?? ??? ?? ?? ?? ????? ???? ??? ???? ?? ??? ??? ??? ?? ?? ??? ?? ?? ??? . ?? ????? ? ?? ??? ? ??? ???? ??? ?? ??? ??? ?? ??? ???? ?? ????? . ?? ???? ????? ? ??? ?? ?? ? ?? ?? ?? ?? ?? ??? ?? ???? ??? ??? ??? ?? ?? ?? ????? ?? ?? ?? ?? ??? ? ??? ? ??? ?? ? ?? . ???? ????? ? ??? ??? ?? ?? ? ??? ??? ???? ??? ??? ??? ??? ??? ??? ???? ?? ?? 1500 ?? 2000 ???. ??. ? ?? ??? ???? ?? ?? ??? ?? ?? ?? ?? ?? ? ?? ?? ?? ???? ????? ?? ????? ? ?? ?? ? ??? ? ??? ??? ?? ??? ???? ????? ? ?? ?? ?? ??? ??? ?? ??? ???? ?? ??? ?? . ? ? ? ??? ??? ?? ?? ??? ??? ??? ?? ?? ? ?? ???? .
?????? about 1 month later:

? ??? ?? ???? ?? ?? ??? (??? ?? ??...) ???, ???, ???? ???- ?? ?? ?? ??. ?? ? ?? ???: ?? ??? ? ??? ???? ?? ???, ??? ? ???? ??? ? ??? ???, ?????? ????, ??? ?? ???, ???, ??? ??’?? ?? ??? ??-? ?? ?? ?? ??? ???? ?? ?? ??? ??, ?? ???? ??? ?? ??? ?? ??? ??, ???, ?????.
???? ????? about 1 month later:

??? ??? ?? ?? ??? ??? ??? ??? – ?? ??? ? ? ??? ?? ?? ??, ?? ?? ??? ? , ? ?? ??? ?? ?? ??? ???? ??? ? ?? ??.
???? ??????? about 1 month later:

?? ?????

?? ?? – ??? ??? ???? ? – ?? ?????. ?? ??? ??????

?? ??? ??? ???? ?? 10 ?? 80 ???.

??? ?? ???? ???? ?? ?? ????.

?? ??? ?? ??? ???? ?: ??? – ??? ???? – ? ?? ????? .

?? ??? ? ?? ?? ??? ??? ?? ?? ?? ? ?? ?? ????. ??? ??? ?? ???? ??? ??? ?? ??? , (? ? ??? ?? ??? ?? ???? ) , ? ?? ?? ??? ? ?? ? ?? ???? ?? ??? ? ???? ?? ??? ??? , ???? , ??? ?? ???, ??? ??, ? ??? ?? ?? ?? ??? ’???? ’.
drums 4 months later:

ldf
??????? ??????? 5 months later:

?? ?? ?? ??? ?????
daniel 5 months later:

??? ?? ? ?? ????? ?? ?? ?? ? ???? ?? ??? ? ??? ??? ? ??? ??
????????? 5 months later:

??? ?? ? ?? ????? ?? ?? ?? ? ???? ?? ??? ? ??? ??? ? ??? ??
itamar 8 months later:

? ?? ??? ??? ?? ?? ??,???? ?? ??? ???? ?? ? ?? ?? ???
Itay 8 months later:

? ??? ?? ?? ?? ??? ???
Eli 9 months later:

? ?? ??? ? ? ?? ???? ??? ?? ??
FLV extractor 9 months later:

so it is
Eli 9 months later:

<a href=”http://www.clickon.co.il/he-il/about/what-is-affiliate-network.htm //”>??? ??? ???? ?? ?? ?? ????
fff 11 months later:

asd
http://www.panelal.co.il 12 months later:

Great blog thanks.
Aluminum 12 months later:

Great blog thanks.
Judaica store about 1 year later:

The best blog
???????? about 1 year later:

the greatest blog ever! i can use the showers here!
????? about 1 year later:

just the greatest
http://www.whiteiphone4transformer.com about 1 year later:

Though white iphone 4 is at high price, but with all-powerful function, we can see lots of people just crazy for the latest white iphone 4.
replica watch rado about 1 year later:

NIC EPOST
????? ????? about 1 year later:

You can see you put effort into your blog.
Deejays about 1 year later:

??
Deejays about 1 year later:

www.deejays.co.il
momo about 1 year later:

http://www.pay-call.co.il
<a href="http://www.presentsimple.co.il">?????</a> about 1 year later:

thanks

presents are fun!
????? about 1 year later:

????
??????? ???????? about 1 year later:

well how did think about this final version?
????? ???? ?????? about 1 year later:

let put and bid on the last thought. huge improvement, which led to a few more significant improvements. Ultimately, so true!
???? ??? ??????? about 1 year later:

Looks really pro, thanks for the info.

?? ? ??? ?? ??? ??
?????? ????? about 1 year later:

looks promising, all the best!
POPO about 1 year later:

looks good thanks ?? ???? ?? ?? ??? ???? ??? , ??? ?? ??? , ??? ??? ??? , ??? ???? ?? ?? , ??? ?? , ??? ? , ?? , ???? ?? ?? , ??? ???? , ?? ?? ??, ??? ?? , ??? ???? , ???? , ?? ??? , ?? , ?? , ? ? , ??? ?? , ??? ?? , ?? ??? , ?? , ?? ?? , ??? ??? , ??? ? , ???? , PVC , ??? ?? ?? ?? ??? ????.
iPhone SMS to Mac Backup about 1 year later:

Would you like to banckup iphone SMS to mac, macBook, macbookPro as .txt files? Now a software iphone SMS to Mac Backup can help you to realize it.
iPad to Mac Transfer about 1 year later:

iPad to Mac Transfer can help you transfer music, movie, photo, ePub, PDF, Audiobook, Podcast and TV Show from ipad to mac freely.
cold room about 1 year later:

Haile Electric Industrial Co. Ltd. is the leading manufacturer and exporter of cold room,axial fan,condensing unit,centrifugal fan,shaded pole motor and refrigeration products in China.
okey oyunu oyna about 1 year later:

Good information.

internette görüntülü olarak okey oyunu oyna, gerçek kisilerle tanis, turnuva heyecanini yasa.
rony about 1 year later:

http://www.partystore.co.il/
Monster Beats By Dr. Dre Studio about 1 year later:

The Monster Beats By Dr. Dre Studio Headphones are classic shoes.You will be proud of owning them. Don’t hesitate!Take Monster Beats By Dr. Dre Pro home! Choose our Monster Beats By Dr. Dre Solo Headphones will make you have an amazing discovery.
christian louboutin shoes on sale about 1 year later:

The article is dull.
Unique Article Wizard about 1 year later:

very nice article ! quite helpful for a newbie like me !
cookies gift baskets about 1 year later:

hmm ,i’m not sure if this is what i’m looking for but anyway this is interresting and could be useful some day,thanks for taking time to write such cool stuff
beats by dre store over 2 years later:

I catch you twice, you refused, so not to face it, no, you should go back after me again, I refuse you, the psychological balance it!” Female choice, said to me: “guy,high quality headphones new design headphones you do my boyfriend right?”
????? ?????? over 2 years later:

? ??? ???!

??? ????
beats by dr dre over 2 years later:

beats by dr dre beats by dre salewhat i’m looking for but anyway this is interresting and could be useful some day,thanks for taking time to write such cool stuff
shanewatson404@gmail.com over 2 years later:

Wow, this is Legal Assistant Job Description really superb info is visible in this blog Director of Operations Job Description that to amazing Accounting Job Description technology and using the wonderful messages for this website. I was searching the great info and providing the nice info in this blog and read Graphic Designer Job Description this info you can never forget this info
Tips For Bowling over 2 years later:

Pleasure is a necessary reciprocal. No one feels, who does not at the same time give it. To be pleased, one must please. What pleases you in others, will in general please them in you.
www.shuttersfix.co.il over 2 years later:

??? ???? ?? ????
christian louboutin over 2 years later:
The professional design make you foot more comfortable. Even more tantalizing,this pattern make your legs look as long as you can,it will make you looked more attractive.Moveover,it has reasonable price.If you are a popular woman,do not miss it.

Technical details of Christian Louboutin Velours Scrunch Suede Boots Coffee:
```
Color: Coffee
Material: Suede
4(100mm) heel
Signature red sole x
```
Fashion, delicate, luxurious Christian louboutins shoes on sale, one of its series is Christian Louboutin Tall Boots, is urbanism collocation. This Christian louboutins shoes design makes people new and refreshing. Red soles shoes is personality, your charm will be wonderful performance.
Spyder Jackets over 2 years later:

http://www.spyderjackets-outlet.net
Spyder Jackets over 2 years later:

http://www.spyderjackets-outlet.net
Spyder Jackets over 2 years later:

http://www.spyderjackets-outlet.net
eynavis over 2 years later:

http://www.eynavis.co.il/
iphone mp3 to ringtone over 3 years later:

iPhone MP3 to Ringtone Maker can convert MP3 to iPhone/iPhone OS 3.0/3.1 Ringtone, even convert any video/audio to iphone ringtone.
Womens Moncler Jackets over 3 years later:

Everyone should have a Mens Moncler Jackets as it is the best outwear against the cold season and is also comfortable and light to wear.
iPhone contacts backup over 3 years later:

It is really a good example that most of the programmer should think about this. If we want to do much better for the code. I should understand it. right? You are very good at this and show us good coding idea.
ralph lauren over 3 years later:

Siècle Série classique américain de los angeles marque Ralph Lauren hommes d’hiver communiqué de 2012, los angeles principale saison des longues de design guy cuir, créateurs intelligents ajouté fermeture éclair dans los angeles couche moyenne de cuir, veste durante cuir cuando longtemps devient instantanément courtroom. Los angeles couleur beige et noir également, put créer united nations design parfait guy scandinaves.http://www.vetementpoloralphlauren2012.net
Ahava over 3 years later:

@ralph, we in Ahava love your work for the American market, It’s definitive and surly worth a glans. objecmentors blog is a great platform for our communication
bladeless fans over 3 years later:

Another Kata: Expression Tokenizer 68 good post15
louboutin sales over 3 years later:

Another Kata: Expression Tokenizer 69 hoo,good article!!I like the post!149
longchamp outlet online over 3 years later:

sdsdasdadsadsadsadsa
Christian louboutin Miami over 3 years later:

The Christian louboutin Miami like watching fantasy and magic movies can try this kind of headphones, exaggerated modelling and magic scene, help you win leadership must be very high.
robdress over 3 years later:

More than satin strapless mermaid dresses a few fox news responders agreed upon with Hayes and thus mutual that they cocktail dresses australia can be calling in the brand new year without a companion. Jasp8019 documented, “Barb, Over beautiful romantic beach crochet wedding dress the past week i have been previously dark reality what quite possibly single and i also don’t have any partner you can make out preference rub about night time. In the past six the latest springs Eves ankle sprain have been finding that screwy all most wives.
birkin hermes over 3 years later:

The two main objectives of having a handbags business website are to bring people to buy these products and earn a substantial amount of income?
????? ????? over 3 years later:

??? ??? ??? – ?? ??? ?? ’ 1 – ??? ???.
girl cosplay over 3 years later:

http://www.outfitscosplay.com/cosplay-catalog/azumanga-daioh-cosplay Deluxe Azumanga Daioh Cosplay Costumes for Sale.Find your favorite characters and cosplay outfits from all the popular anime and games.

Mentor	twitter id
Uncle Bob	unclebobmartin
Brett Schuchert	schuchert
Michael Feathers	mfeathers
Bob Koss	bob_koss

Another Kata: Expression Tokenizer 76

Comments

Blog Search

Follow us on twitter

Categories

Blogroll

Syndicate