Test Strategy – a handy checklist

More guidelines than actual rules
From XKCD 1287


As part of trying to educate other people at work about testing strategy, I’ve been thinking about it a lot recently – in particular because I keep seeing similar issues and often struggle to concisely get across.  While I’m still working on that thinking (hopefully post(s?) to follow), one thing I keep coming back to that’s really useful is one of the checklists from the Rapid Software Testing Appendices – “a set of considerations designed to help you test robustly or evaluate someone else’s testing”.  It’s not the be-all-and-end-all, but a lot of the test strategies I come across miss thinking about a bunch of the stuff here, so using this is a good starting point when coming up with your strategy.

(All kudos for this list => The RST creators)

Project Environment

  • Mission. The problems you are commissioned to solve for your customer.
  • Information. Information about the product or project that is needed for testing.
  • Developer Relations. How you get along with the programmers.
  • Test Team. Anyone who will perform or support testing.
  • Equipment & Tools. Hardware, software, or documents required to administer testing.
  • Schedules. The sequence, duration, and synchronization of project events.
  • Test Items. The product to be tested.
  • Deliverables. The observable products of the test project.

Product Elements

  • Structure. Everything that comprises the physical product.
  • Functions. Everything that the product does.
  • Data. Everything that the product processes.
  • Interfaces. Every conduit by which the product is accessed or expressed.
  • Platform. Everything on which the product depends (and that is outside your project).
  • Operations. How the product will be used.
  • Time. Any relationship between the product and time.

Quality Criteria Categories

  • Capability. Can it perform the required functions?
  • Reliability. Will it work well and resist failure in all required situations?
  • Usability. How easy is it for a real user to use the product?
  • Charisma. How appealing is the product?
  • Security. How well is the product protected against unauthorized use or intrusion?
  • Scalability. How well does the deployment of the product scale up or down?
  • Compatibility. How well does it work with external components & configurations?
  • Performance. How speedy and responsive is it?
  • Installability. How easily can it be installed onto it target platform?
  • Development. How well can we create, test, and modify it?
  • (I’d explicitly break out from that last one
    • Testability.  How well can we put the product into the state we want to put it into for testing?
    • Diagnosability.  How well can we tell what state the product is in?
    • Debugability.  How easily can we work out what has happened?
    • Maintainability.  How easily can we fix issues in or enhance the product?)

General Test Techniques

  • Function Testing. Test what it can do.
  • Domain Testing. Divide and conquer the data.
  • Stress Testing. Overwhelm the product.
  • Flow Testing. Do one thing after another.
  • Scenario Testing. Test to a compelling story.
  • Claims Testing. Verify every claim.
  • User Testing. Involve the users.
  • Risk Testing. Imagine a problem, then find it.
  • Automatic Checking. Write a program to generate and run a zillion checks.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s