Enlightment, planning and convictions

Sometimes, learning new things is scary.

You've just learned something you find very important, acquired a new skill and you ask yourself:

"How can I have worked without understanding something so crucial during all this time?"

Well, another more useful question usually pops up in my head:

"Why was I not able to pick this up before?". This is certainly more useful because it opens up the possibility to pick it up faster next time!

Recently, I got the feeling that I learned very important lessons about the raison d'être of agile methodologies.

The cards array enlightment

I remember standing in front of our cards array contemplating the next iteration planning and the result of our last planning game. I was thinking: "now I know why I was not so effective as a project manager for planning the activity of the project" (better late than never!)

Planning, yes but why?

I always had difficulties for planning software activity, and was even beginning to think that development was something like the butterfly story introducing any chaos theory presentation: a beat of wings and milestones go away.

Many of my attempts to get more insightful plannings failed. Here's a set of recurring issues:

• how can the development planning stay in line with the global roadmap (especially if it is very "ambitious" but also a moving target)?
• how do you plan bugs?
• how can you make accurate predictions regarding the delivery date?
• how do you fight developers natural optimism ("well, 2 days should be ok for the parser")?
• how do you plan non-development time (meetings, holidays, illness,...)?

Moreover, every "aggressive planning" I saw was the recipe for repeated failure. And the effects on the team are disastrous: "Whatever we do, we are late". But "late" relative to what? A release date set by a manager and approved by a developer muttering to himself "I'll do what I can". So what's the real use for a planning?

A manager answer included: "setting a challenge to motivate the developers". This is the "aggressive planning" theory. A really counter-productive one, since it leads to the "We're always late" syndrom, which is soon followed by the "We're poor developers" consequence.

I had my own variant on aggressive planning: "Watch what you do and register time passed". I thought that doing this will drive individuals to be more conscious of "unproductive" time and get rid of it. I was wrong. A developer can chase "unproductive" time only if he has decided to, not if he has to fill a time report precise to the hour (I have even read about minute-precise time report, where bathroom time is to be precised!). And then, as a mean, he can watch his time spent.

So I figured out there must be better reasons for doing plannings (instead of the "I develop to my best abilities, you pick the product when ready" approach).

One of the reasons is resources management. If time-to-market is a crucial issue to the Product owner, then he may wonder "how many people can I add to the project to get it faster?". Once you keep in mind the Mythical man-month essay by Fred Brooks (and this funny anecdot), you can still try to add people to reach the project optimal size.

One other reason, is some kind of control over what is developed. Plan your activity, do the job. If along the way, the plan is not respected, this may be interpreted as a warning. However, if the meaning of the warning is that your plan was not accurate, you have not gained much.

And maybe the most evident reason is the need to communicate dates to people outside of the project: customers, sales representatives, consultants, investors,...

Planning revisited

After our 6 months effort to implement the full agile practises stack (we already had continous integration, unit testing, iterations), I now understand why this works:

• you deliver software in a time-box manner and focus on priorizing customer value, having both short-term control (the iteration) and mid-term control (the milestone)
  

• you have a very simple measure for productivity: velocity. You globally measure the capacity of the team to deliver useful functionality. Meetings occur, bugs occur, obscure technical work occurs but it is not counted as such

• estimations are more reliable because of the planning game. Indeed, estimating a task is better done by a bunch of people having different experience and knowledge about it (I used to do that on one-to-one with the "concerned" developer - which is not a relevant concept in eXtremeProgramming anymore)

• estimations are also more reliable because of the frequency of the planning games (every 3 months)

By the way, I want to emphasize all the benefits of the planning game:

• developers have the opportunity to understand precisely the customer objectives
• the customer can get a better grip on development risks and difficulties
• the team is really empowered and feels responsible for the overall project

Let the team manage itself

The last point is what stroke me in front of the cards array: many brains are better than one.

• the team is much more efficient at organizing the project than the project manager
• the team is much more efficient at defining and estimating the tasks
• the only way to get the maximum productivity is to trust the team members and let them set high-productivity standards. No management indicator can ever imply high productivity. You cannot measure commitment, smartness, collaboration

The consequences for a project manager seem pretty clear to me: you are only here to help the team manage itself.

How to do better next time?

Thinking about that, I concluded 3 things:

• I should have experimented more. Some agile practises seems evident. Continuous integration is one of them (spend machine time instead of developer time). Some other are not so intuitive: why use physical cards to track development tasks? It is not until you hold one in hand that you can feel why

• Empower the team. This is the brain number effect. I also find some relations with Michel Crozier's theory regarding the fundamental liberty of any worker to gain power around him. This also means that team management can never be an easy path

• I should live up to my convictions. For instance, I was not strong enough on emphazising why continuous integration should always be green, no matter what (this one I experimented)

Don't give up on your conviction

We all make mistakes, learn more or less painful lessons. However, when we learn something the hard way, we'd better not forget about it. What we learn become our convictions and the remaining questions have to be experimented.

I recently read a blog entry of an experienced developer analyzing the reasons behind his software projects failures. The whole entry is worth reading and wondering about. But one specific thing was worth remembering for me right now:

And if you decide to make changes, have the courage to go 100% with your gut. I've failed more than once when I watered down my convictions in order to appease dissenters. The only thing worse than evangelizing change and failing is looking back and realized you might have succeeded if you'd held firm on your convictions. What a waste!

I won't be caught twice on that one.