Tag Archives: Agile software development

Business, Old Favourites, Project Management, Software Development

Old Favourite: Adaptive Budgets? “Pull” the other one!

Leave a comment

This was originally posted on my old blog on 10th April 2010

Recently, I wrote about my views on using and estimating with task-cards. I highlighted that tracking progress with burn-up/down charts showing effort completed/remaining is not a true measure of progress, especially if we subscribe to the idea that we measure progress with working-software.

I also highlighted how tasks “horizontally slice” a “vertically sliced” story.

This inspired an article on infoq by Mark Levison. Since its publication, there have been several comments.

There are some specific points I’ll be answering on infoq, but some general points are more easily answered here…

True measures of progress
One of the key points I was trying to make in my first post (linked to above) is that if “working software” is our measure of progress then tracking task completion is not consistent with that.

One of the problems task-hours are trying to solve is to provide a visible indication of progress. It’s also something that many people find easier to understand. The idea of relative-points estimation seems to baffle many people when they first encounter it.

In my original post (linked above) I referenced an approach, blogged about by Jason Gorman, outlining a solution to providing a visible indication of progress that is compatible with the idea of measuring progress with working software. It also works more seamlessly with the solution of measuring throughput trends of the team (e.g. with story points) so that the team can estimate how much can be done in the next iteration.

Value Points & Task Points
Value points were mentioned in the infoq comments. Value points are solving a different problem and don’t help the team estimate what can be done. The value of a story isn’t an indication of its complexity. Value, combined with complexity points, can help determine priorities. Arlo Belshey explains some interesting views on this.

Prioritisation is, in my view, one of the few reasons to place some sort of estimate on value & complexity. Unfortunately, it seems to be often used to establish a contract with the team for when something will be done and apply pressure when the estimates turn out to be inaccurate.

Non-estimating pull systems
My preference is to use such estimates only for prioritisation… after that, things just take as long as they take. There is some benefit in tracking accuracy trends so that we can learn from them, however, spending too much time on these things simply slows down the process of getting things done. Interestingly, I have not yet seen the business be quite so passionate about evaluating whether their value estimates were accurate when the product makes it to the real world. Funny that.

Instead, I prefer to leave estimation behind once we’ve prioritised things and pull stories or customer valued work items through the system without using previous estimates to predict their completion date. With the way that most budgets are determined and allocated, this idea isn’t compatible with the way much of the business world works. For pull systems that eliminate estimation as a means of predicting the future to work (such as Kanban), we need a more adaptive approach to budgetary spend. Business needs to find a way to adapt budget allocation more frequently, perhaps as frequently as monthly, perhaps more frequently still. The business now needs to look at how it can respond to change rather than focus on following a budget-plan.

Business-world: now it’s your turn
Budget holders now need to be more agile. We, those who evolve the implementation of the ideas of the business, have responded to their demands to be able to respond to rapidly changing markets and provide that competitive edge. For the more mature implementation teams, the hindrance now is no longer how we make the products, but the business culture of inflexible and predictive budget allocation.

Agile, Business, People, Project Management

My Tack on Effective Change

4 Comments

One of the key characteristics of how we coach our clients’ teams is that we help them start from where they are and introduce small, frequent changes that help them progressively achieve their goals.

Each incremental change is driven by a problem the teams recognise they are facing. We then help to find a small change they are happy to introduce as an experiment. This change must be a solution or a step towards a solution to the identified problem. Sometimes this experiment will be limited to a single iteration. Sometimes it’s limited to a single user-story in a given iteration.

Based on the outcome of the experiment, the team decides what to do next. They may continue the experiment, introduce the change beyond the limitations of the experiment or even abandon that change and try something else.

This has two distinguishing factors.

1. Solving Problems: We focus on solving problems not introducing solutions

2. Continuous Improvement: Delivery continues, becoming increasingly productive

1. Solving Problems: For example, we’ve often been contacted to ‘introduce TDD’ or ‘transition a team to Scrum’. Usually, we find that behind these solutions are problems that our client instinctively understands are there but articulates them as these kinds of solutions. Sometimes they want to reduce the amount of re-work, or get early visibility of achievable scope within a time-constraint. Other-times it’s a solution for solution sake.

By solving the problems the people in the organisation recognise as holding them back, they are bought into each change. Sometimes our job is to help them recognise the problems. Retrospectives are a great way of achieving that.

2. Continuous improvement: When helping an organisation to solve the problems they face they expect large and significant change. Instead we help them make a continuous series of small, team-driven changes. This allows them to continue to deliver with the net effect of gradual and continuous improvements in productivity.

We generally find ourselves taking this approach because for most of our clients, big-bang change is not practical simply due to the impact on productivity. We can understand this using the Satir Change Model (explained very well by Steven Smith).

In this simplified and slightly adapted representation, you can see that with a large change there is a period of net-negative productivity. Once the improvement starts to take hold, performance increases take productivity back into the black.

On rare occasions when an organisation’s software reaches critical mass (it’s more expensive to change than to replace) big-bang changes might be necessary. We’ve worked with a client that had exactly that problem. They took the brave step of ceasing delivery while they redefined how they did things and coached their teams in how to do it. This worked out positively for them. After about 6 months of intensive coaching and 3 months of implementation they delivered a revenue generating system to replace something that had been previously worked on for 18 months and had remained incomplete.

Most organisations would find it difficult to take that step and need to continue to deliver whilst transitioning from how they work today to how they want to work in the future. It is for this reason that we find ourselves employing small incremental changes, eliminating the things that add friction to their process – one problem at a time. Doing this with small experiments helps the organisation limit the risk and achieve continuous delivery and continuous improvement.

A way of understanding how this works is with the same Satir Change Model. If we take the curve shown above illustrating big-bang change, shrink it so that each change represents a much smaller alteration to working practices and then repeat, what we end up with is a much healthier looking series of changes.

Yes, each change can slow productivity temporarily but the impact is dramatically reduced and quickly offset by the improvements gained.

The area under the graph is productivity. If the changes are small enough there is never any period of net-negative output.

Another reason this works well is that it reduces the amount of cultural resistance that you have to deal with. Instead of a lot of resistance from many people for complex reasons, you tend to have only a few people resisting the change. Because the change is driven by a need to solve a recognised problem you have more people in favour of the change. Because change is frequent, people get used to change. In fact, they become experts at it and no longer resist the mere idea of change.

Commercially, it also gives you much more flexibility and allows you to adapt your vision continuously as you learn more.

Julian Davies, a developer at one of our clients, likened this to sailing. To travel from point a to point b in sailing you do so in a series of direction changes, called tacking. This is necessary when point b is against the wind.

You could, theoretically, do this in two long legs with one direction change. However, this leaves you vulnerable to changes in wind-direction potentially taking you significantly, perhaps even catastrophically, off course. Tacking is generally done with a series of small diagonal legs where each leg allows you to adapt to the changing wind. These frequent opportunities to read the context, i.e. the wind and currents, means that we will only ever be slightly off course with plenty of opportunity to recover.

Julian likened it to agile development in general but I think it applies equally to what I’m talking about here today.

With big-bang changes, what happens when we’re in the deepest trough of the curve and an opportunity arises? It is much more difficult to change course. With small changes we can put the latest experiment on hold to capitalise on that opportunity. When the wind changes on our projects, say when someone leaves or half the team is ill with a winter flu bug, we can quickly respond to those changes.

Acknowledgements: Thanks to Andy Palmer for reviewing this article and providing the snappy title.

Software Development

Monsters, Names, Pot-Roast & The Waterfall Model

11 Comments
“Antony” (without the ‘H’) is the anglicised version of Antonius. In victorian times (there or thereabouts I’m guessing), among those wishing to appear oh so intelligent, gossip spread that the spelling of “Antony” was wrong… For, so they would say, it is born of the greek word “anthos” (meaning “flower”) – oh dear… so many poor children with misspelt names… 

Despite being completely wrong, the world forgot of my name’s etruscan origin and spelt it with an ‘H’… This misinformation established itself through the eras so much so that, today, the de-facto spelling is “Anthony”. It has even found it’s way into the American pronunciation of the name as: “An-thon-ee”.

Waterfall development has something in common with this story… somehow, through misinformation, what it once was has been warped, into something else.

The key difference is that Waterfall is now increasingly represented as was originally intended. Unfortunately for me, my name is not…

 

Monsters & Legends

Some might think that the Waterfall Model is an approach to software development, first explained (but not named) in Winston Royce’s 1970 Paper “Managing the Development of Large Software Systems” (PDF), but they could be wrong…

Somehow, it seems to have become something else… it became the way (many) people thought software should be developed… the norm for software ‘professionals’. Years of anecdotal failure followed and Waterfall became a legend – told time and again much like a scary camp-fire story… The enemy of effective software development… A monster that will consume all the resources it can, spewing out nothing but documentation, rarely concluding in working software – at best 20% of the time.

This negative view, to what was once the de-facto approach to software development, is actually far closer to Royce’s original words on Waterfall than many seem to know…

 

The Truth & Technology

In Royce’s original paper, he shows a progression of activities, that came to be known as the waterfall model.

What we rarely hear of is Royce’s original words on the subject:

“…the implementation described above is risky and invites failure.”

Further to this, Royce goes on to explain that the reason that this cannot work is because there are too many things we cannot analyse up-front:

“The testing phase which occurs at the end of the development cycle is the first event for which timing, storage, input/output transfers, etc., are experienced as distinguished from analyzed. These phenomena are not precisely analyzable. They are not the solutions to the standard partial differential equations of mathematical physics for instance.”

He explains that we need feedback loops. He goes on to warn of (a conservative) 100% overrun in schedule and costs:

“…invariably a major redesign is required. A simple octal patch or redo of some isolated code will not fix these kinds of difficulties. The required design changes are likely to be so disruptive that the software requirements upon which the design is based and which provides the rationale for everything are violated. Either the requirements must be modified, or a substantial change in the design is required. In effect the development process has returned to the origin and one can expect up to a 100-percent overrun in schedule and/or costs.”

Some of Royce’s strategies, like “Involve the customer” and obtaining early feedback, have lived on in modern (Agile) methodologies. Beyond that, we should remember that his specific recommendations on how to solve the problems of a waterfall model were all based on the technology of the time.

 

Pot-Roast & The Cost of Change

In 1970, computing was much more expensive than it is today. In those days, changing software was far more expensive than changing pictures and words on paper. It was also much harder to express your design in a human-friendly way in the programming languages of that time. As a result of these and other factors, documentation was a major part of how Royce tried to solve the inherent problems of the Waterfall model.

 

Technology, tools & thinking have moved on and our documentation no longer needs to be static. It lives. It can breath. The specification can automatically verify that the implementation does what we said it should do (e.g. as in BDD Specs or ATDD/TDD Tests). Modern programming languages allow us to express the design and our understanding of the domain far more clearly, negating the need to first detail our thoughts on paper in natural language. We simply don’t have to cut the ends off that pot-roast anymore.

 

Only now, at the end…

Waterfall, thanks to the popularity of Agile, has gone from something I was shown at school as “how software is developed” to being seen in the light that it was originally presented – how software should not be implemented.

This is despite those who still profess the legitimacy of Waterfall and those still shocked and surprised when they hear of Royce’s own words against the monster he unintentionally created.

As for my name, I hold out little hope for change. I doubt that the world will use my name as it was originally intended and so I have resigned myself to needing two domain names… one with an ‘H’ in it, and the correct one without – I wonder which one brought you here.