I attended a lunchtime set of lightning talks at ThoughtWorks yesterday (11/11/09) in downtown Chicago. It was great, but surprisingly they had trouble nailing down compelling arguments for adopting Agile practices. I wanted to help them by adding a business perspective to “trust me, you’ll like it better,” and summarizing a few other topics of the day interpreting them in terms of my own experience.

They also dispelled the belief by some audience members that Agile is a radical new way of doing things, and it correctly came out that many practices were developed previously, even on the mainframe. I am not taking anything away from the leaders in the Agile community who have popularized many best practices, encouraged collective adoption and developed specific ways of incorporating them into coherent software engineering frameworks.

I realized I myself ended up doing a variant of extreme programming (XP) without knowing anything about it through application of common sense optimizations and best practices. Hopefully my notes here will help people understand why certain engineering practices are advocated by XP. One day I ended up being a development lead with no knowledge and a blank slate and this is what happened…

Code Quality Beyond – “It Works”

Erik Dornenberg mentioned that code should be easy to change and ThoughtWorks CEO Becky Parsons mentioned that you show pay attention to testing when writing code. My own mantra is “good quality code should be easy to change, easy to test and easy to debug.” The relative priorities of each is determined by business needs.

Test Driven Development

In my environment, I often was given requirements that were simple technical specifications. For example, I had to ensure that some value followed a naming convention. The best way to test that the code met the specification was to encode the specifications into the unit tests and then write the code so that it passes the tests. How else would you do it? The optimization here is that I didn’t want to waste my time writing code that didn’t pass a unit test, so I wrote the test first. Possibly I found this out by trial-and-error.

Automated Testing

Duh – I had 300 unit tests for my codebase. Am I going to run each of them manually? We did have sufficient complexity that I worried that one change might break something seemingly unrelated, so I made sure all unit tests were run before promoting the code for deployment. So, no changes could be deployed without all unit tests being run against the code.

Continuous Integration

I’ve also done a lot of consulting in the application lifecycle management space and, we didn’t get too deep into developer practices. But, since we were implementing version control with automated build, we always introduced continuous integration – we just didn’t have that nice tidy term. One day, I hope to thank Martin Fowler (who gave one of the lightning talks) for popularizing/coining the term so I don’t have to fumble with a longer explanation. Unfortunately, when I now say “continuous integration,” I’m met with a blank or embarrassed stare (“I feel I should know what that is, but don’t”) so I still have to go through the full explanation.

On my own team, I wanted to know about integration problems as soon as possible and I wanted to ensure unit tests were written for code changes by other developers. So, I had unit tests automatically run prior to code check-in. A failing or missing unit test meant the code could not be checked in. Well, I lost that battle and had to relax the enforcement to mere warnings. I even had a unit test generator to make it easy to pass.

BTW, this was all Perl and you can have integration issues there just as in C# or Java. Integration problems often have to do with namespace and interface changes, and those issues come up in Perl also. Perl also gets compiled prior to runtime, so you can also break the Perl build. There is just not a separately executed build step (ok, ‘perl –c’ – maybe I’ll try that next time).

Pair Programming

We sort of half-implemented pair programming and code reviews because the engineers were not familiar with Perl best practice coding. They were also not familiar with the architecture and standard patterns I invented and had in my head alone. Yes, another nasty real world situation. Had I introduced actual “pair programming” it would have been better than me looking over someone’s shoulder and trying to convince them to use best practices to no avail.

Had I introduced actual pair programming, I would have had to immediately answer to the project manager who would want to know why it was taking two people to make each change instead of one (sure there are twice as many changes, but…). This was the hottest topic of the day at ThoughtWorks, but no one in the audience articulated the question that succinctly.

Neal Ford in his talk, told us that, with pair programming, it took 15% longer to code the changes, but it resulted in 15% fewer defects. Then came “trust us, that’s better.” You were almost there Neal. We need cost and timeline justifications for the management to accept pair programming. The one thing I’ll add to the below analysis is that probably most of the 15% defects would get caught in pre-release testing for non-pair programming, but let’s say one would get through to production.

Cost Justification for Pair Programming

Find out the cost of a production defect. At the low end, it is a few $10k’s, at the high end it’s millions. Let’s say with pair programming, we prevent one defect that would cost $50k plus a lot of stress.

Then you have on top of that the cost of finding the other defects in QA, repairing them and possibly impacting your timeline. That’s got to be worth at least $50k for a typical business app.

Now compare with the 15% extra time it takes to make the changes in pair programming. That’s 15% extra time for two programmers over let’s say a take a two-week iteration. One of our customers uses $65/hr to estimate the cost of salary AND all the IT resources consumed by a developer, so let’s use that. The cost of extra development time is:

80 hrs/developer * 2 developers * 15% * $65/hr = $1560

That is nothing compared with $100k worth of problems and sill worth it if I’ve wildly overestimated the cost of the defects.

Timeline Justification for Pair Programming

To calculate the cost of the extra test and repair work, you really needed to know how long all that would take and what the extra effort is because time is money. Al l you have to do is take 15% of 80 hours (12) and compare that to the time it takes to test, repair and re-test. It is probably worth investing the extra time time for pair programming.

Interleaving Design and Coding

When I became a dev lead, I was supposed to do design and let the junior members code, but I could not do one without the other. I couldn’t articulate why, but I did both and it worked out. Now, we hear that is a best practice, and to wait as long as possible before doing the design and design while coding. I’d still like a well-articulated argument about why it works.

Effective Communication

Communication breakdowns are a common reason for project failures. Features of Agile methodologies such as the stand-up meeting, well-defined artifacts and having testers and business analysts interacting very closely with developers are ways to facilitate communication. Discover (the credit card company) has a Discover Lean initiative that puts business analysts and developers in the same room, accomplishing the same thing.

Summary

I was also happy to hear the experienced ThoughtWorks folks espouse an adopt-what-works methodology rather than follow a specific methodology religiously. That is very good advice in my view. After all, I was very successful with IXP – Ignorant Extreme Programming.