As part of trying to educate other people at work about testing strategy, I’ve been thinking about it a lot recently – in particular because I keep seeing similar issues and often struggle to concisely get across. While I’m still working on that thinking (hopefully post(s?) to follow), one thing I keep coming back to that’s really useful is one of the checklists from the Rapid Software Testing Appendices – “a set of considerations designed to help you test robustly or evaluate someone else’s testing”. It’s not the be-all-and-end-all, but a lot of the test strategies I come across miss thinking about a bunch of the stuff here, so using this is a good starting point when coming up with your strategy.
- Mission. The problems you are commissioned to solve for your customer.
- Information. Information about the product or project that is needed for testing.
- Developer Relations. How you get along with the programmers.
- Test Team. Anyone who will perform or support testing.
- Equipment & Tools. Hardware, software, or documents required to administer testing.
- Schedules. The sequence, duration, and synchronization of project events.
- Test Items. The product to be tested.
- Deliverables. The observable products of the test project.
- Structure. Everything that comprises the physical product.
- Functions. Everything that the product does.
- Data. Everything that the product processes.
- Interfaces. Every conduit by which the product is accessed or expressed.
- Platform. Everything on which the product depends (and that is outside your project).
- Operations. How the product will be used.
- Time. Any relationship between the product and time.
Quality Criteria Categories
- Capability. Can it perform the required functions?
- Reliability. Will it work well and resist failure in all required situations?
- Usability. How easy is it for a real user to use the product?
- Charisma. How appealing is the product?
- Security. How well is the product protected against unauthorized use or intrusion?
- Scalability. How well does the deployment of the product scale up or down?
- Compatibility. How well does it work with external components & configurations?
- Performance. How speedy and responsive is it?
- Installability. How easily can it be installed onto it target platform?
- Development. How well can we create, test, and modify it?
- (I’d explicitly break out from that last one
- Testability. How well can we put the product into the state we want to put it into for testing?
- Diagnosability. How well can we tell what state the product is in?
- Debugability. How easily can we work out what has happened?
- Maintainability. How easily can we fix issues in or enhance the product?)
General Test Techniques
- Function Testing. Test what it can do.
- Domain Testing. Divide and conquer the data.
- Stress Testing. Overwhelm the product.
- Flow Testing. Do one thing after another.
- Scenario Testing. Test to a compelling story.
- Claims Testing. Verify every claim.
- User Testing. Involve the users.
- Risk Testing. Imagine a problem, then find it.
- Automatic Checking. Write a program to generate and run a zillion checks.