Mitigating Requirements Risk With Agile Processes

Abstract
Software projects are often late because the wrong thing is built and needs to be changed. In this article I discuss the problem of requirements risk in bespoke software projects and how to mitigate the risk of building the wrong thing.

Current Practices For Meeting Requirements

Consider the following statements:

A) The goal of development is to deliver software which meets the requirements of the users. So we should spend most of the time coding and building software.

B) Coding is really only about 10% of the job. The rest is making sure you write the correct code. So spend less time coding and more time planning and modelling requirements.

If A is assumed true we hear statements from Mr Manager like “I want to see people at their desks with their heads down coding. They must produce code. If they aren’t coding, they aren’t working.”

If B is assumed to be true we hear statements from Mr Modeller like “We need to spend time doing lots of analysis and design up front to make sure we build the right thing and it is designed for future requirements.”

So, are A and B true or false?

The actual answer is that it was a bit of a trick question. The initial statements in A and B are correct, but the inferences that follow them are flawed which is why we hear the common statements from Mr Manager and Mr Modeller. In a table they look like this:

Statement	Inference	Opinion
The goal of development is to deliver software which meets the requirements of the users. (TRUE)	Spend most of the time coding and building software. (FALSE)	If Inference = TRUE Then: “I want to see people at their desks with their heads down coding. They must produce code. If they aren’t coding, they aren’t working.”
Coding is really only about 10% of the job. The rest is making sure you write the correct code. (TRUE)	Spend less time coding and more time planning and modelling requirements. (FALSE)	IF Inference = TRUE Then: “We need to spend time doing lots of analysis and design up front to make sure we build the right thing and it is designed for future requirements.”

Because the inferences are false the opinions are also false. So, why are the inferences false?

Well, the answer is a simple one: We don’t know the system requirements at the start of a project. No one does. The customers don’t and the developers don’t. We need to stop deluding ourselves that we do.

We Don’t Know The Requirements

Let me justify my last bold statement. The majority of software requirements are discovered after the first release of a system. That is why so much is spent on support, maintenance, and enhancements. It’s also why it is often delivered late, because the requirements change. To prevent missing the deadline the developers often start beating the customer over the head with the requirements document with statements like “well, that’s what you asked for”.

Show me a software developer or customer who knows precisely what the system requirements are from the start and I’ll show you a stock broker who can predict accurately the future of the market.

Still, when the project fails (because the wrong thing was built) or the stock market crashes we all become experts after the event. The IT people proclaim that “It’s the users fault, as they said they wanted A, so we built it. Now they want B.” The stock broker says “Well, yes, the reason for the crash was x, y and z.” Then these experts convince everyone they saw it coming and charge people absurd amounts of cash to predict when it will happen again! Then it does and they come out with some other poor excuse. The stark reality is that there is an unknown, called the requirements. All we need to do is accept the fact that they will change and evolve during the life cycle of the project. We then have to mitigate the risk of getting them wrong.

The Doctor Who Doesn’t Diagnose

A little anecdote which my good friend and consultant Jason Gorman told me the other day at our clients site.

A business manager goes to see a doctor who just happened to be a software developer in the past.

Doc:	“Morning Mr Business Man, what can I do for you?”
BM:	“Well, I’ve got this pain in my leg.”
Doc:	“Ah, I see, and what would you like a prescription for to help it.”
BM:	“Ah, I don’t know, you’re the Doctor.”
Doc:	“Yes, I know and I’d like to help you. What would you like to solve your problem?”
BM:	“Ah well, I’ll try some Prozac, I’ve heard that’s quite good. I read about it in a glossy magazine.”
Doc:	“Okay. [Doc writes out prescription for Prozac.]”

The following week…..

BM:	“I’ve been taking the Prozac for a week and I’ve still got the pain. Hasn’t help at all.”
Doc:	“It’s all your fault. You asked for Prozac. Now this time get it right. What do you want?”

And so on until the patient (project) dies.

The Doctor is failing to diagnose the problem and the patient, the business manager, knows nothing about medicine. We have to accept that developers know nothing about the business and what the problems are, and that the business knows nothing about software and what it can do. Ask them what there problems are, not how they want you to solve it.

Mitigating Requirements Risk

Consider the following:
1) No one knows the full system requirements at the start of a project.
2) We can only truly know the requirements after we build the system.

Chicken and Egg situation a bit isn’t it? It also suggests that we should hack away building any old thing and then throw it at the users and then ask them what they think! I’m certainly not going to suggest that though! So, what can we do? Well, below is a list of development principles, many of which you will find adopted in the Agile development Community, particularly in Extreme Programming.

Stop Talking About Systems And Focus On Business Value: We need to stop asking cusotmers what systems they want, since they don’t know systems as well as the developers. We need to start talking more about business processes. We need to start asking them what their problems are and establish where we can add business value. We need to diagnose the problem before building a solution.

Understand The Problem: To make sure we build the right thing we need to understand the problem. We very often build the wrong thing (or something that is cool) because we don’t understand why we are building it. If we were closer to the problem we would be able to make sensible suggestions about potential enhancements based on business value, rather than on ‘this would use XML, cool’. As Duncan Green says “To understand the real problems in the business the development team needs to get closer to the heartbeat of the business.” Talking to the business face to face is far more effective than producing an enormous requirements document full of Chinese whispers.

More Communication: To get closer to the business requires frequent open and honest communication. The best form of communication is face to face. After that it is the phone, and documents are way down the list. We should spend more time speaking to the customer and less time writing documents. Remember, our goal is to build software, not produce a list of documents that no one will read, and then bash customer over the head with it.

Feedback Early And Often: We acknowledge that, as developers, we know next to nothing about the business problems compared to what the business know themselves. We therefore need to get feedback as early and as often as possible, which is why we build simply and quickly in very short iterations.

Iterative Development: To confirm we are building the right thing we need to build it and show it to the user. So, we need to build quickly and simply. So, we build using small iterations (2-3 weeks) which enable us to get immediate feedback from the user.

Keep It Simple (YAGNI): We need to build the simplest thing that will work so we can get something out quick and get immediate feedback. YAGNI comes from Extreme Programming and stands for ‘You Ain’t Gonna Need It’. We should not build stuff they haven’t asked for because we would then be guessing the requirements.

Just In Time Development: In the manufacturing industry they use JIT to cut down the costs of holding stock. What is the point of building stuff for which there are no requirements and holding them in stock? It costs money to build and maintain code that no one asked for. It is also very likely that YAGNI because the requirements change throughout the development. In the same way we should do Big Model Up Front, we shouldn’t do Big Framework Up Front.

Refactor The Code: We are not doing Big Model Up Front (BMUF). We are building the simplest thing that will work so we can get feedback as early and often as possible. Therefore we need to ensure the internal design of the system does not rot. Refactoring means changing the internal structure of the system without changing the external behaviour. To do effective refactoring you also need automated tests.

Automate The Testing: Manual testing is ad-hoc, non exhaustive, and developers don’t do it. If you automate your tests you spend more time designing and building new functionality than you do fixing existing functionality. If we spend less time fixing stuff we have already built we can then spend more time building new stuff and getting it to the customer to satisfy the feedback, early and often target. See my articles on Test Driven Development and Developing Automated Tests Using NUnit and VB.NET .

Practice Writing Tests First: We should write our tests before we write our code. Then when all the tests pass we are done. The tests will feed into the design processes so we build something that works and meets the requirements. We also end up with a regression test harness as an end product. We can also apply this concept at the analysis stage by doing walkthroughs with the customer of their improved business process with the proposed system requirements.

It’s The Content, Not The Style: We often need to write things down, such as headline requirements and models. The content of what we write is more important than the format in which it is written. There is no point spending further time typing it all in to a word processors or modelling tool if we do not have a requirement to ‘make it look pretty’. Travel light is the motto here.

Summary

The biggest risk to project failure is getting the requirements wrong and building the wrong thing. To mitigate this we need to focus more on diagnosing the business problems rather than asking the business for their solution. We need to accept we don’t know the requirements from the start and that they are going to change. We need to mitigate requirements risk by having better communication with the customer and building early and often to get rapid feedback.

Happy developing.

Dave Chaplin