Agile and the second chasm: history does repeat itself

In Computer science, Software Engineering on November 8, 2011 at 5:05 pm

Kent Beck posted an interesting essay on the Agile Focus blog back in February. I think it’s a really insightful, and important statement from one of the Agile elders (sorry Kent, but accept the fact that we’re getting older). Although this is eight months later, I just read it due to a re-tweet of the link by one of the people I follow. A downside of the Web and the age of instant communication is that there is an exponential explosion of content, much—maybe most—of it is simply noise.

I’m glad that I came across this post though. It made me think hard about what he’s saying. I have found that over the years, I agree with Kent much more than I disagree with him. I remember being on a panel at conference back around 2002 on XP, Agility, and process in general and it seemed that Kent and I were on the same side, even though I was there officially representing Rational and the Rational Unified Process team.

The other panelists, all Agile consultants, were promoting dogmatic adherence to all of the XP practices. They indicated that they forced all team members to do all of the practices all of the time when they were engaged to help teams us XP. Well, if you look at XP, that’s what you’re supposed to do. But, that doesn’t always work. You need to use some common sense.

Kent and I were suggesting that the specific set of practices and how dogmatic their application should be appropriate, not just a blind application from a book or some other source. In short, it would be better for the team to succeed by adopting a handful of practices and using them in a way that suited the team than it would be for them to turn all the XP practice dials up to 10 and fail.

During the Q&A period, Mary Poppendieck made the comment that except for Kent and I, the other panelists sounded like the process police. I found this quite funny. A significant part of the Agile movement, besides trying to discover better ways of delivering software, was to combat RUP and other process products that were deemed to be too restrictive and dogmatic (they also had a significant portion of the market). Getting into the Agile camp allowed consultants who worked individually or in fairly small companies gain traction in places where they might not otherwise have been able to compete. Certainly large companies were reluctant to get away from the IBM Rational security blanket. They may not always succeed, but they followed the conventional wisdom of “you can’t go wrong with IBM.”

So, here we are more than ten years after the Agile Manifesto was signed. Are things different? Well, yes and no. Certainly we have added many new tools to our software engineering toolbox. We look at software development differently in certain areas. We are tend not to inflict heavyweight methods when lighter ones will do. In fact, we are probably guilty of erring to the opposite end of the spectrum now and not formalizing things when we probably should. After all, we don’t want people to think that we’re not Agile!

One might argue that many of the changes we’ve seen would have occurred naturally, without the Agile movement. But, however the changes have come about, I think we have a better set of tools—intellectual and otherwise—to use in our craft. What I don’t think has changed to a large extent is the ability for software developers and managers to use common sense when deciding how to work. They still want someone to tell them what to do and how to do it.

At the end of the 1990s, the RUP was sold as a process framework that you could customize to your team, the type of project you had, and the overall organization. Customers who did were usually quite successful. Those who thought they had to do everything in the several thousand pages of advice in the RUP had some spectacular failures. Of course Rational, and later IBM, was happy to send consultants to your company to help you figure out the right configuration. That is, someone would tell you what to do, how, and when.

Today, Agile consultants come and help you adopt Scrum (certified Scrum Masters), or XP, or Lean, or Kanban, or … . It doesn’t matter what. We don’t want to think and reason about what’s best for us and our project teams. We want someone to tell us. Perhaps we think it will allow us to avoid taking responsibility for the outcome of our projects; never mind that a cornerstone of Agility, or any reasonable methodology, is reflection by the team and adjusting the process. That seems like too much (non-productive) work. We can hire someone to tell us what to think and how to do our jobs. After all, they’re the experts, the consultants who have the experience.

So, we’re right back where we were ten or twenty years ago. We let people tell us what’s good for us. We don’t think. We let others do that for us. We’re always riding the crest of the next wave, be it XP, Scrum, Lean, or whatever. If we’re current, we must be doing the right thing. That may be true only if the current thing actually applies to what you’re doing. In 1999, Alistair Cockburn talked about Methodology per Project. I think this essay went unnoticed at the time. That’s a pity. I think there are some real nuggets in what he talks about. Of course he’s a consultant trying to convince you that his Crystal family of processes are right for you. Simply put, every project and project team is different. We don’t manufacture software, because each program is different. If we write the exact same code multiple times then we’re idiots. Ours is a craft. We produce one-of-a-kind products. If our products are one-of-a-kind, then maybe the way we produce them should be as well. That’s the key.

If we don’t start smartening up, we’re going to be in the same place ten years from now. Oh, there will be new methods and tools, but we won’t really know how to use them. We’ll hire someone to tell us what to use and how to use it. Same old, same old. Unless we start thinking for ourselves, reflecting upon our experiences, and taking our own destiny in our hands, we’re doomed as a profession. As the world’s appetite for software increases, we’re going to sit back and hope someone can show us how to do better.

When students get out of my software engineering course is that they are able to evaluate tools and methods and pick the right ones for their projects without someone else telling them what to do.

  1. Hi, Gary! I’m flattered that you’d think my post was written by Kent, but I wouldn’t want him to get blamed for any of my mistakes, so I should point out that “Agile’s Second Chasm” is mine alone.

    • Sorry William. It wasn’t clear from Kent’s blog, or I missed it. Anyway, it was really insightful.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s

%d bloggers like this: