The agile experiment (part 1)

The most interesting and eye-opening talks at EuroPython 2007 were probably Arlo Belshee’s talks on agile methodologies and his team’s experiments and results in that regard. The following blog posts summarizes what I learned from listening to his talks, doing his XP workshop, discussions with him, and digging up related information on the web (such as a paper and a podcast). I strongly believe this could be the next phase in how we develop software, and I hope I’ll be able to whet your aptetite for change as well :)

First, let’s get in the right mindset. Isn’t it amazing how much we don’t know about software development still?

  • Why are some teams orders of magnitude more productive than others?
  • Why is there such a huge variation in the quality of software developed?
  • Why do some people love software development, while others hate it, and how come so many only discover which way they feel after they finish their software development degree?
  • What’s the best methodology or process for developing software, and how come some methodologies say exactly the opposite?

I think this stems from a couple of issues:

  1. Software development is still a relatively young discipline
  2. We probably had the completely wrong starting point, thinking that software is anologous to construction engineering
  3. Despite wise men’s warnings, we’ve been looking for The One True Process, the Silver Bullet, that we can push on any team and take away the pain

It’s become more and more clear that software development is much more of a creative discipline than everyone thought at first, that it’s more about people adapting and learning than it’s about following a pre-determined process. And, since it’s about people, and since people are different, there isn’t any one process that’s going to work for all. So, if we can’t just adopt a process and become super-effective, what can we do?

After listening to Arlo it’s a bit obvious: we have to treat software development like a continuous science experiment: track what we care about (metrics), make changes, measure the impact, repeat ad nauseum.

Although we regularly or irregularly discuss process problems, this isn’t nearly enough:

  • We don’t objectively measure the impact of changes. We need metrics.
  • We’re reactive, focusing on fixing things, instead of proactive, focusing on experimenting.
  • We’re not brave enough with the changes we do. We should be extreme so we can see extreme impacts.
  • We don’t talk about process frequently or regularly enough. We don’t learn quickly.

Safe is for weenies

So, as long as we have this continuous improvement built in, no matter what process we start with, slowly over time we’ll inevitably gravitate towards the most optimal process for the team we’re in.

How do we get started?

Metrics

Before we start making big changes, we need to make sure we will be able to tell if the change is good or not. We need to identify what we care about, and track it in some objective way. We need metrics.

Arlo’s team identified three aspects with simple metrics associated with them:

  1. Productivity. A team is productive if they can get the same amount of work done more quickly. If the way we estimate work remains constant, we should be able to see changes in how much work we get done. Arlo suggests not using hours but instead something more abstract; they use something called story points. If we use hours, let’s at least differentiate these hours from actual hours. We’re interested in precision and predictability, not accuracy. In any case, however we express it, what we’re talking about is the team’s Velocity, how much work they can get done in a sprint.

  2. Defects. Arlo’s team were, XP style, using index cards. They used red cards for tasks that were not planned, either caused by unexpected dependencies, defects, or very large refactorings. They measured quality as the red card rate; how quickly did they occur.

  3. Code debt, or technical debt. The idea is that if you don’t reach the quality plateau, you’re not paying the full price for the feature the first time around. You are instead in debt, and will have to pay the price in future refactorings and inefficiencies. This is measured as the number of refactorings that took longer than 15 minutes to do; the thinking is that you’ll normally do refactorings all the time, but if you find yourself having to do bigger ones, that’s a sign of smelly code. Code debt takes longer to discover than the above.

I think these are a good start.

The “meta process”

So we’ve already dropped the idea of the One True Process. That doesn’t mean there isn’t a process, it just means the process will have to change from team to team, from project to project, and over time.

At a minimum, though, for this to work as a science experiment, we need to build into the process steps for regular reviews and changes. Agile methodologies like XP provide a good starting point. They define a cyclical, iterative, process that consists of at least these steps:

  1. Planning. We plan in detail only what we’re going to need this sprint. Scrum advocates planning more and further ahead, whereas Arlo advocates not having any plan besides what we’re going to do this sprint. I can see and feel the cost of planning lots of stuff that are far into the future, so I think we can at least move more towards Arlo’s ideal. It takes a lot of courage to not plan.

  2. Doing. In this phase XP, and especially Arlo, are doing things a lot differently from the way we currently do them. I’ll get into much more detail about this in the future blog posts.

  3. Retrospective. This step we’re mostly not doing at all. The idea is that we review the metrics from last iteration–e.g., was the changes we made good or not, or don’t we have enough data yet–and we make changes. Changes should be radical in order to force a difference, e.g., not having standups, having 5 standups, doing pair programming, not doing code reviews, working from home, meeting with client, not meeting with client, and so on. Anything goes.

  4. Demo. Show what was accomplished to the client. This we’re already doing to varying degrees inside of Exoweb. Direct connection with the client is very valuable in fostering mutual understanding.

If you’re still doing XP two months after you started, you’re not doing XP.

Just a note on standups: Arlo’s team currently does multiple standups every day to help the team focus and re-energize.

What did Arlo learn?

So, Arlo’s team has been doing this for several years, mutating the practices of XP, with some astonishing results. Other teams have also tried out their practices, with mostly good results.

So, I think we should definitely put these high up on the of changes to try out.

I’ll discuss each one of them in separate blog posts. This is part 1.


Share it on...
del.icio.us  Digg it  Netscape  Newsvine  reddit  StumbleUpon  Yahoo MyWeb  

4 Responses to “The agile experiment (part 1)”

  1. Bjørn Stabell 白熊 » Blog Archive » Promiscuous pairing (part 2) Says:

    […] Bjørn Stabell 白熊 thought exception, brain dumped… « The agile experiment (part 1) […]

  2. Bjørn Stabell 白熊 » Blog Archive » Team-owned tasks (part 4) Says:

    […] Part 1: The agile experiment Safe is for weenies […]

  3. Bjørn Stabell 白熊 » Blog Archive » Arlo’s agile experiment summary (part 5) Says:

    […] Part 1: The agile experiment Safe is for weenies […]

  4. Bjørn Stabell 白熊 » Blog Archive » Least qualified implementor (part 3) Says:

    […] Part 1: The agile experiment Safe is for weenies […]

Leave a Reply


tracker