Requirements, specificaion and design are these three things that happen before you get into writing any code (or testing it), and from experience it’s tricky to properly disentangle them. This matters, because what the customer actually wants is something that fulfils the requirements that they want, but the natural way for us developers to think about things is in terms of design, and so can end up focussed on the wrong thing!
Here’s a simple rule of thumb that lets me spot when I (or someone else) have mixed these up.
- Requirements are WHY we are doing what we’re doing.
- Specification is WHAT we’re doing.
- Design is HOW we’re doing it.
For example a statement that says “The customer needs a link at the top corner of every page on their website that takes you back to the home page.” sounds very much like a requirement. But putting a link at the top corner of every page on the site is a WHAT, not a WHY – so that’s specification pretending to be a requirement. The actual requirement is probably something like “The customer wants to be able to get back to their home with one click.” or perhaps even “The customer wants their site to be easily navigable.”
Have a go when you’re creating or reviewing any or all of reqs, specs and designs. Does this help? Does it not? Are there obvious exceptions? Let me know…
This is really a sister post to Ask Why?. If as a tester you’re given a task, or you’re briefed about a feature, or you’re learning about a new protocol, or anything. Stop. Assuming that you haven’t just been assigned custodianship of the entire universe, you’re starting with a box. The box has boundaries (in fact the better the person assigning you the task is at their job, the clearer those boundaries usually are) and you’re being pointed at the stuff inside the box and not worrying about the stuff outside the box.
Start off by stepping outside your box and looking in from the outside. Why is it here? Who cares about it? What do they want it to do? Is it self-consistent? Does it appear to behave sensibly? Just spending half an hour outside looking in and asking obvious questions gives you the chance to cheaply find the really big bugs, both in the product and your testing plans.
And these are exactly the sort of bugs that you won’t find by “testing harder”. They’re the ones where
- there’s a fundamental assumption in the spec that isn’t correct (“Won’t American cooks be confused by an oven that only goes up to 250 degrees?”)
- there’s been scope creep and extra function that isn’t needed (“The customer needs a cheese grater, why have we created an everything-grater with multiple detachable configurable blades?”)
- the testing you’re about to do is all invalid because you’re missing context (“Why is all our testing being done in GMT for this computer chip designed especially to work in Australia?”)
Do the step-back. Ask the questions. Save yourself days of time!
The Goal is a book about management. It was originally written in the ’80s, I think intended to change how western manufacturing thought about managing manufacturing processes – and it certainly has.
It’s written as a novel. It’s actually well written in a thriller style – though the problems and context are all about a plant manager attempting to balance and save his plant, career, family, etc. This makes it really easy to read, while walking you through a lot of interesting ideas. You might find it a bit slow or strange if you’re used to more modern dry punchy books, but it’s a great starter to get you interested if you’ve not thought much about it before.
In particular, I found two parts of it very interesting, as related to software development.
- Most of the book is about understanding what’s important in a manufacturing process and thinking about pulling work through the plant rather than pushing it. While the understanding the book gives doesn’t directly relate to sofware development, it’s the same basic thinking that supplies Kanban and it’s a really good intro to why Kanban and similar methodologies are useful.
- The last section of the book looks to wider management and more abstract management problems and ways of dealing with them. These aren’t very different between managing software development and anything else.
I also recommend it as an easy starter into reading books that cover more than just raw technical information!
It struck me recently that there are two clear modes that we operate in when testing – and our behavior in those modes is different. This post is my attempt to define them – so that I can be more aware of what I’m doing and work on them differently.
Mode 1 – Searching
When you’re Searching for bugs, you’re expecting that something’s out there. You’ve done your background thinking and decided on an area to look at or a rough approach – but you don’t know where any individual bug actually is.
When you’re in searching mode, your actions are to cover as much ground as possible and to find any inital indication that there’s something you might want to investigate further. This means it’s good to
- change lots of things at once
- keep summary notes of what you’re doing and where you’ve been
- defocus, distract yourself, use breadth-first search techniques
- use System 1 – i.e. fast, reactive, emotional and assumption based approaches.
Mode 2 – Tracking
You swap into Tracking Mode when you catch a glimpse of something that’s not quite right. You now know there’s something funny going on, and you’re working to get from symptoms that there might be a bug in the area to having the bug boxed up and delivered.
When you’re in tracking mode, you’ve found the signs of a problem and now need to hunt it down. Your actions are about focusing in from symptoms to bug – or bugs, be ready to find a whole nest of the blighters! This means it’s good to
- change one thing at a time
- keep exact notes of what you’re changing and what you’re keeping the same
- focus and use depth-first search techniques
- use System 2 – i.e. slow careful, rational thinking.
To be clear, these modes aren’t “better” or “worse” than each other; you need to use them both to hunt down those bugs. Just be aware which you’re doing as you do it.