Work

Approaches to Developing Software

Home > Work > Approaches to Developing Software

Introduction

Someone starts a career writing software. After 10 years or so, she finds that doing certain things seems to yield better results than not doing them, or doing other things. She writes them down, so that others starting a career writing software can shortcut their learning.

That's what a methodology is. It's a blueprint of the things that should be done to ensure success. There are many methodologies. Some people think that some work better than others. Other people think none of them work. Some get a little more nuanced, and think that doing agile badly is cheaper than doing waterfall badly. This is quixotic - wasted time is wasted time, and technical debt (agile done badly) is still technical debt (Waterfall done badly).

Today methodologies fall roughly into these two categories - Waterfall or agile. Waterfall is seen as slow, cumbersome and expensive. Agile is seen as fast, nimble and cheap. And yet neither quite measure up to their claims.

I'm going to briefly describe and then compare the strengths and weaknesses of both. I'll finish by pointing out that success depends on the experience of the team and their techniques (good practices), rather than choice of any methodology.

Let's start with Waterfall:

Waterfall

Waterfall is composed of an ordered set of phases. Phases cascade down what many believe are the essential components of the software development process. The order of these phases is arranged to yield the greatest chance of success.

Weaknesses

Waterfall doesn’t describe development techniques at all – it describes a project management approach. Most of the content is devoted to talking and reporting about the job.
Waterfall can lead to the production of large amounts of documentation that may not be of use to anyone.
Requirements are almost always incomplete, and some missing. Incomplete requirements cause downstream costs to increase exponentially.
Producing tangible output takes too long. From the user's perspective, the team disappears into a room and doesn't come back out until it's all been built.
Isolated phases produce communication gaps. Each boundary between phases is an opportunity for misunderstanding. Analysts, for example, gather requirements and move on. Architects rely on the quality of documentation produced by the analysts to design the solution.

All of these weaknesses pale in comparison to this one:

Correcting errors is expensive. Waterfall facilitates moving downstream through phases, in sequence. Moving back upstream is costly.

Strengths

Whether we want to admit it or not, Waterfall has exceedingly compelling advantages:

It makes very efficient use of specialised skills, because they generally correspond to phases. This results in good horizontal integration, where a specialist applies a skill during a phase of a project, and then moves on to apply the same skill to the same phase in another project.
Its sequentially-phased nature makes it easy to manage.
Waterfall produces documentation.
Waterfall is predictable.

Agile

In agile, activities are bound to other activities, and not to phases. Phases seem to be done concurrently. Agile is, in essence, a one-phase approach. This is why eXtreme Programming, scrum, and so on seem like undisciplined hacking. However, it’s not that at all. It is a way of developing software when not only the solution, but also the problem is unknown.

Weaknesses

Agile's weaknesses are a little more difficult to pin down. Other than a lack of context or concrete guidance, the agile manifesto contains no apparent weaknesses. This left me with my observations and experiences, and looking at specific agile methods.

The manifesto has a footnote which states that the four values on the left are preferable to the four counter-values on the right, even though the given counter-values have value. This is often interpreted as meaning the items on the right are bad, and must be derided and ignored.
Agile has an extensive collection of practices. It fails, however, to describe the context in which those practices work, or don’t.
The same lack of context encourages dogmas, bigotry, and claims of universal applicability.
Responsibility for project failure is abdicated to project sponsors, on-site customers, users, or not employing one of its practices.
While agile is undoubtedly suited to environments in which the problem being solved is ambiguous or hasn't been articulated at all, agile is often used as an excuse to ignore even trying to articulate the problem.
Architecture and design is actively discouraged. Refactoring code or re-writing system components later can cause downstream costs to increase exponentially.

Some method-specific disadvantages:

Kanban is just a scheduling system. As such it does no more for development methods than Waterfall.
Scrum makes use of cross-functional teams (where all roles are committed to a project from start to finish) but, because it doesn’t define an end state, doesn’t offer predictability beyond the current sprint (iteration).
In every method that uses them, user epics and stories are almost always incomplete, and some missing entirely. Incomplete stories cause downstream costs to increase exponentially.
User stories are often recorded on paper cards, and hardly ever placed under change control. Lost and missing stories, and stories altered without a change control process inevitably lead to conflict within the team and between the team and customer.
Agile produces little, if any, documentation.

All of these weaknesses pale in comparison to this one:

Product owners (in Scrum) or customer representatives (in eXtreme Programming) are non-technical stakeholders that have the authority to make technical design decisions. While never a good idea, this has particularly bad consequences on architecture (software design) and security.

Advantages

Agile addresses many of the shortcomings of Waterfall. It provides, for example, early and constant visibility of progress in the form of working code. It therefore provides transparency.
Agile allows the project to adapt to changes quickly.
Scrum makes use of cross-functional teams (where all roles are committed to a project from start to finish) meaning that communication within the team is excellent.
It has practices explicitly intended to improve software quality.
It's suited to environments in which the problem being solved is ambiguous or hasn't been articulated at all.
Agile produces little, if any, documentation.

The Role of Experience

Changing context for a minute, let's talk about martial arts.

Karate is a martial art. It is also a sport, with strict rules and competitions. Krav Maga is street fighting - no rules, and no competitions. I've been doing Krav Maga for over four years, and think it's the dog's bollocks.

Saying that Krav Maga is the best is interesting. Not because it really is the best, but because that statement demonstrates two important things:

It demonstrates consistency - my desire to be (and appear to be) consistent with what I have already done. Once we've made a choice, we encounter personal and interpersonal pressures to behave consistently with that commitment.
It ignores reality - my girlfriend, a 3rd now 4th Dan Karate black belt, has been practicing for 27(!) years. Whether Krav Maga is technically better than Karate or not is immaterial - she is simply better at martial arts than I. No matter how brutal a counter attack in Krav Maga is, she's so fast that there's no way I can block her attack (I know because we spar).

The same philosophy applies to software development processes. An experienced team using Waterfall is more likely to succeed at a project than an inexperienced team using agile processes and techniques. The reverse, of course, is also true.

Conclusion

All of the successful projects I've worked on succeeded in spite of their methodologies, rather than because of them.

Agile has no upfront design. An experienced developer anticipates downstream requirements and makes the correct choices at the start. Similarly, an experienced developer will create a prototype or proof of concept during the design phase of a Waterfall project to demonstrate his thinking, thereby reducing risk, and saving money in downstream phases.

Experienced developers, whether agile, waterfall or somewhere in-between, will write (or insist on) a written specification. It doesn't matter whether the specification consists of stories, use cases or a MoSCoW list. Because when things go wrong (experience tells us they will), specifications become the team's only sense of "truth". This is critical when a team scales.

Methodology can be helpful or harmful to development because without context, no single rule is universal. Choose a methodology that's appropriate for your sponsor, your project, the team, and your your environment.

< Back to Work | ^ Back to top