Fundamentals: The first line of the Agile Manifesto

How much wisdom is in the first line of the Agile Manifesto?

Well, what is the first line? I ask, because I all too often see people think that it’s “Individuals and interaction over process and tools”. That not the first line. That’s the first value, and I’d argue that missing the first line means that you’ve totally missed the deeper context for all of Agile thinking.

“We are uncovering better ways of developing software by doing it and helping others do it.”

That’s the first line. If you doubt me, check AgileManifesto.org.

Before you think I’m just being pedantic I’d like to walk through this sentence and really bring out its glory.

  • We”: this is something that’s done collectively, not alone.
  • are”: it’s done in the present, in other words all the time. This is not an invocation of past insights, it’s a reminder that you need to be present, in the present, conscious to what’s going on right now, and not relying on what might have happened in the past.
  • uncovering”: not inventing, or selling, or theorizing, or proposing. By being present, by paying attention to what’s going on here and now moment by moment, we can uncover what is emerging. Paying attention to the work will reveal how the work is actually working.
  • better”: not best, but better. In a creative field there is no best but there could always be better.
  • ways”: what ways? We’re not saying. We can’t. We don’t know. We are uncovering them. Agile has no answers! Sorry about that. If we thought we had the answers – the ways – we’d be selling you something that had worked for us in some situation we were in. Not your situation now. So we might achieve compliance if we had enough authority, but can you smell the hubris in that? The whole point is that we’re doing something creative. Something we’ve never done before. If we had done it before, we’d be using that! The trap is thinking you have done something before when you haven’t and relying on the “ways” that worked back then. The word “playbook” – I’m looking at you! And you, “standard operating procedure”! Those ideas are useful, even necessary, but they’re simply not sufficient in an rapidly changing world.

Where was I?

  • of developing software”: the authors of the Manifesto were at the time only thinking of software. But by taking out these words and replacing them with “working” we have a much more broadly applicable philosophy emerging. For developing software of course this all applies. But starting to work consciously, always being present to what’s emerging, that’s helpful in any field. It’s dogmatic to insist this only applies to software, and it’s closed-minded to think, for example, that “hardware can’t be Agile”. Remember “ways”? Uncover some better ways for hardware, why not? For accounting? Why not? For facilities? Why not?
  • by doing it”: the ways are being uncovered by the actual practitioners. Not theoreticians. Now theory and academic work is essential, I grant you. It can shift the paradigms of thought in which the work occurs, so it can actually accrue as “experience” not just “time served”. Ten years thinking the same way is not ten years experience. So think, and do. But theory on it’s own, no matter how fascinating, is ultimately of no value unless it’s applied.
  • and helping others do it”: which rounds us back to doing this not just collectively but also collaboratively. You get more done working with people than you do alone. More than that, it speaks to a deep principle of life: you can get whatever you want if you help enough other people get what they want. Remember where the money comes from.

This is the first line of the Agile Manifesto:-

“We are uncovering better ways of [working] by doing it, and helping others do it.”

It goes on “Through this work we have come to value-”

Now go read the values and principles and see if they don’t taste different than they did before you read this article. Delicious, eh?

Sublime and ridiculous

Here’s the sublime: http://arlobelshee.github.io/AgileEngineeringFluency/Stages_of_practice_map.html?  Arlo Belshee’s clarity is just dazzling.

And here’s the ridiculous http://lafable.com/  You Agilists should find this highly entertaining, and you may be surprised when you find out who’s behind it.

I’m clear that oddly, the message in the comical one is what’s missing from the serious one.  Arlo’s graphic is fantastic, but it is dazzling, and dazzled you might miss something.  What is it? Take a look, what do you think?  How would you express that message?

Why are estimates nearly always low?

Why oh why?

How often is an estimate correct? Why do we seem to under-estimate so often? We can’t always blame the pointy-haired boss.  Can we?

2015-02-16 Estimates DilberWell, no we can’t.  Really, we could have some sympathy for those pressuring us to higher certainty and lower costs.  They’re just trying to reduce risk, even if the risk is of looking bad. Maybe they’re stuck in a world of cost accounting, unaware of its impact or alternatives.  If a project has overrun, and if all you know is detailed estimation, then it’s easy to assume that you can’t have done enough detailed estimation.  So there’s an urge to do more of it!

Doing more estimation is a waste of time and energy.  Let me show you what is going on here.  It is something simple and fundamental.

Let’s Estimate Something

Here’s some work we’re estimating. Let’s consider everything that could be known about this piece of work.

IMG_1212

First, there’s some of it about which we have a high degree of certainty.  We know what we know about this.  Let’s call this KK – Know what we Know.

IMG_1213

Then there’s the rest, the stuff we Know we Don’t Know. Call this KDK. This must have a degree of uncertainty, by definition. We could manage the uncertainty if we buffer the estimate.  With a mature and wise customer, we could provide a range.  The risk is that we assume the high end is more realistic, and those employing us turn the low end into a deadline.

IMG_1214

So, we’re done.  That’s our reasonable estimate.  Quick! Commit!

Easy, tiger!

Wait!  We’ve missed something.  It’s not a reasonable estimate.  As it stands, it’s pretty much a guarantee of failure.

What are we up to here? We’re developing something new.  If we were developing something that wasn’t new, we wouldn’t have to estimate it, would we?

If it’s new, there must be parts of it that we Don’t Know that we Don’t Know (DKDK), by definition.  I mean – it’s new!

Here’s my question: will DKDK lead to more  work or less work?  (Hint: it’s MORE work!  Duh!)

IMG_1215

Well, here’s a grain of solace.  Inside DKDK is DKK: what we Don’t Know we Know.  Some stuff  turns out to be very like something else we’ve already done, and we hadn’t realized it until we started work on it.  This will account for those occasions when the estimate turns out to be high – sound of heavenly trumpets!  But since we usually remember what we know, these occasions are not the norm.

IMG_1216

The point remains: it’s all pretend until we start work on it and start realizing – making real – what’s actually involved. Nothing is certain until– when?– until it’s actually finished!

The truth is out there

Take a look at Steve McConnell’s classic work on the Cone of Uncertainty.

 

Note that the broad end of the cone, at the time of producing the estimate, gives a range of accuracy of the original estimate of from 25% to 400% of the actual. 

The Standish Group’s CHAOS report shows this clearly.  Here’s the table of Time Overruns from the report.

Time Overruns  % of Responses
Under 20%  14%
21 – 50%  18.30%
51 – 100%  20.00%
101 – 200%  35.50%
201 – 400%  11.20%
Over 400%  1%

Look at this as a histogram:

image

Thanks to DKDK the work more often ends up running long.  Sometimes, sure, it’s early thanks to DKK.  Only 20% of the time does the combination of KK and KDK prove roughly accurate. 

Over to you

This is why the wise thing to do is turn the iron triangle upside down and declare that the team is fixed, the time is fixed (by sprints), and the scope is variable.  This isn’t just wise, it’s because there is no such thing as a fixed scope!  The scope of anything that hasn’t yet been created is unknown, and so must be variable.  It’s all pretend!

Now, do I need to explain the genius of relative size estimation using story points, limiting work in progress, and reducing cycle time?

The only sensible thing to do is to estimate using a scale that is course grained, recognizes that the larger something is the more uncertainty there will be, and is local to those doing the work. Hence Story Points and Relative Size Estimation.

It’s only sensible to limit the amount we do to the smallest possible amount so we can check with our customers if it’s actually useful to them quickly, and adjust course accordingly. Continuous planning makes sense.

It’s only sensible to minimize up front detailed specification and estimation, as that’s the point when we know the least. Understand what is meant by Last Responsible Moment.

So give an estimate, sure.  Estimate the whole thing.  But it is unprofessional and irresponsible to then allow yourself to be held to a fixed scope in an Agile world.  Don’t do it.  You’ll only disappoint your customer when the intention was to delight them.

Comments, as ever, are more than welcome.

In my estimation…

I’ve was asked some time back to be a panel member on a discussion about estimation.  The other panel members are all business analysts.  The audience is going to consist of business analysts and project managers.

Following a planning call with the organizer and other panelists, I realize that I’m dealing with some folks who are used to estimation being a quest for certainty, regardless of the fact that it’s an uncertain world. 

To prepare myself, I’ve been thinking of an analogy I can use to explain agile thinking about estimation.  Here’s what I’ve come up with:

Scenario The First

You’re going to play golf with some friends on a course with which you’re all familiar.  How long will it take to play each hole?  How long for the entire round?

Can you hear people answering already “It depends…”?

Scenario The Second

You’re going to play golf with some friend on a course with which none of you are at all familiar?  How long will it take to play each hole?  How long for the entire round?

Can’t you just hear people saying “Well, we could guess if we looked at a plan of the course” or “We wouldn’t really know until we’d played a few holes”.  Right.

Scenario The Third

You’re going to play gold with a group you’ve never met, at a course at which none of you have ever played.

… The Fourth

…this time not even on a course, but across country!  You have a map, but it’s not clear if the map is current.

…The Fifth

Now, back to the unfamiliar course with your buddies, but this time you have only four hours to play.  How many holes could you complete?  How about if you were only allowed a budget of seventy-five shots, how many holes then?

How about if I told you to play exactly eighteen holes, no more, no less, and they all had to holes in one, or you lose your bonus this year?  On three courses.  At the same time.  I’ve got a buddy who’s no longer a project manager as a result of roughly that scenario.  Which is one of the reasons I’m serious about agility.

Anyhow, that’s the analogy I ended up using.