When is a Defect a Defect?

So I was having a quick look through my CodeProject emails, and amongst the articles I opened to browse was one titled “It’s Not a Bug, It’s…” by Dalip Mahal. Now this article has some good content, but I felt that there were some areas where a response was worthy – and I decided (for once) to create a blog post with a bit more depth than I usually do…

So, where to start? Dalip starts by discussing what makes a defect a defect, and as Dalip mentions it, defect is a better term than bug. Although sometimes defects appear to gather a life of their own, defect has a more professional and clearer context compared to bug.

People don’t respond to bugs with the same recognition of importance as they do defects, the word defect makes people understand that unequivocally it is an error that needs to be fixed, not a feature or other re-definition that people may use to ignore the issue. Dalip then goes on to define a defect exists when the system behaves differently than the requirements specification. So far there is nothing Dalip says that is wrong (in fact this is true of almost all of the article) and having worked in systems engineering and specification for quite a few years I have a natural tendency to agree with what Dalip states, we often under-specify or poorly specify systems. However there is one line that made me stop and think -

Many organizations do not create sufficiently complete requirements before starting development, either because they don't know how to capture requirements properly or because they don't have resources capable of capturing complete requirements. 

It’s that word ‘before’ that concerns me, and I’ll explain why in a minute, but before I can do that I felt that probably we need to clarify what we mean by the term requirement.

Requirement – an Ambiguous Term

This is a real bug bear with me, one of the prime things a good systems engineer does while creating a set of requirements is to ensure that each atomic item is not ambiguous (a difficult skill) and yet the term itself appears to be ambiguous.

So what is a requirement, most people tend to think of written example “The system shall…” statements, and a pre-conceived decision about the level at which requirements turn into design, for example I have worked with teams who consider statements of customer need as the requirements and anything more detailed is design, others have Customer Requirements, System Requirements, Architectural requirements in a more complex hierarchy. Typically with complex systems I would argue that capture of customer requirements and system requirements and then properly applying good requirements management to those layers is enough. However, what about other considerations, such as scenario based methods, SysML / UML, use of requirements databases / repositories, for agile teams are stories the same as requirements?


 That last question is the one I want to bring to prominence for a moment –what about agile teams?

Can we learn about requirements management from Agile?

For me it’s a simple yes, but there are a few layers of why so let’s visit some things that are expected in a good story. I’ll kick off with the INVEST criteria, created by Bill Wake.

  • Independent
  • Negotiable
  • Valuable
  • Estimable
  • Sized / Small
  • Testable

So are any of those applicable to a requirement? Obvious answer is yes – all of them, although for the traditional requirements author valuable and sizable may be difficult depending on where in your requirements hierarchy you are. This is a tricky part of writing requirements (and writing stories), achieving something that is atomic and has its own intrinsic value that can be developed in a short space of time is hard. (Difficult enough to frequently require some time spent in training and coaching). A traditional SE may not be used to the idea of a requirement being developed on its own in the same way as a story is, and there are some requirements that arguably cannot be developed on their own but need to be understood iteration over iteration as we develop.

Another great thing that stories have, which should be applied to requirements is acceptance criteria – this gives a clear view as to whether the requirement (or story) can be tested, it gives everyone a common understanding of the need and helps clarify where there could be ambiguity in the requirement. It is covered by the T in INVEST, but it’s a clear need for a requirements author – what value is an un-testable requirement? However, while there is a clear relationship between stories and requirements are the two equal - I think not, there are clear areas especially about non-functional requirements or system level items such as performance that make better sense as requirements and not as stories.

So let’s get to the trickier part of this then, what else could a requirements management expert learn from the story method used by many in agile? The second item I would like to bring up is timeliness and change. Requirements change, always, on every project I have worked on in differing industries requirements have changed between the start and end of the project. Looking back at Dalip’s statement they I quoted we then need to not consider the word ‘before’ but the word ‘sufficient’. Too many requirements / too much detail too early and we will end up in a pile of churning requirements. Can we look at the idea behind creating a backlog and mirror some of that in our requirements – i.e can we ensure that we deliver the most important items to the customer first and then the less important items. Can we work such that the highest priority items in the backlog should have a much more in detail set of requirements while the lower priority items have just enough to ensure that we are not going to commit architectural suicide in our product?. The answer is yes we can, if we can then allow our requirements authors to adjust what they are doing, they can engage with the development teams and product owners to gather feedback on the early requirements before they create the later ones. The requirements can be adjusted as we make design decisions during the development process – this reduces the risk of a large requirements effort that ultimately gets changed, allows the requirements author to ensure they have time with the development teams, adjusting the requirements as they go.

So what about that Defect then?

The key thing though is again noted by Dalip, in that agile does not mean ‘no documentation’ it means we place a value on the correct level of documentation, we don’t want our system under specified (the phrase Dalip used), we want our specification to be dynamic so that it can cope with the expected changes during development. We want to use the most appropriate style of requirements, so for many  teams it may be that a succinct set of use cases, with only non-functional requirements held in documentation  is superior to traditional requirements documents.

This is important because most systems are under specified (if they are specified at all :-) ). Code can only be considered a defect if it differs from the specification.  We call defects undocumented features because we know that the problem is that the requirements were never written

So Dalip’s article is a good read, for many who have no concept of requirements – it’s a must read, but I don’t believe in a ‘requirements phase’ – yes you need to do more at the start of the project but don’t try to create them all and remember requirements evolve – the art is to manage that change well.

As for ‘undocumented features’ we all know that one phrase translates as ‘oops we missed something’ – probably worthy of a completely different article :-).