sample use case templatesample test plan templateexample project plan template
ICS 125: QA Plans
QA Plans > Who, What, and Why?
- Who: Usually, the lead QA engineer authors the test plan. He/she
must work with the project manager, development team, and QA
- What: A "QA Plan" is a plan for quality assurance.
- Sets goals
- Outlines a strategy
- Identifies actions to be done
- Identifies resources needed
- Schedules activities, allocates budget
- It is basically a subset of the overall project plan
- The test suite(s) and results are separate from the plan
- Why: QA plans are needed to coordinate development activities to
achieve quality. There are many, many possible quality goals and
ways to try to achieve them, so a written plan is needed to make
sure everyone is working along the same path.
QA Plans > Contents
- Identifying information: Project, release, etc.
- Introduction: Background, summary
- Quality goals
- List and describe quality goals
- Reference materials: Links to requirements, design,
- Resources: People, machines, and tools to be used in testing.
- Choose and describe QA activities to be done
- Try to assure that quality goals are met
- Matrix of goals vs. activities
- Risks: What quality goals will not be assured
- Plan of action:
- What will be done, by whom, and when
- Links to checklists, test suites, etc.
QA Plans > QA Activities > Reviews
- Basically, perform reviews that focus on specific artifacts or quality goals:
- Requirements review
- Prototype review
- Design review
- Implementation review
- Documentation review
- UI review
- Security review
- Performance/Scalability review
- Daily or automated reviews:
- All changes to a release branch must be peer-reviewed
- All developer work must compile before check-in
- All developers must subscribe to cvs@PROJECT mailing list
- Nightly run of automated style checker
QA Plans > QA Activities > Testing Approaches
- Exploratory testing: Just get the product and poke around.
Useful if QA team needs to gain familiarity before forming a detailed
plan. Useful for verifying that a product is ready for testing by
- Ad hoc testing: Manually exercise the product to just see if you
can break it. Ideas from test case design can be applied "on the fly."
- Structured testing: More systematic manual testing. Testers
work through a detailed manual test suite.
- Automated testing:
- Record-and-playback: test through the GUI or web interface with a simulated user
- Programmatic testing: write test code to run against product code
- Black box / specification-based testing: Test only the visible
UI or specified API, do not make use of any knowledge of the
implementation. Try to cover every specified requirement.
- White box / implementation-based testing: Design the test suite
using knowledge of the product implementation. Try to cover every
part of the implementation.
- Regression testing: Verify that solved problems remain solved.
And, testing that fixes do not introduce unintended changes.
- Smoke testing / Quick tests / Nightly tests: Automated test of
selected features that can be run often. E.g., run by developers
before they do a commit.
- User acceptance testing: Final, high-level test of the entire
system to see if it is acceptable to users. Think in open-ended,
- Beta testing: Usually ad hoc testing by limited outside people.
Beta test programs must be managed to get many results.
- Early access: Actual usage by broader outside people
QA Plans > QA Activities > Testing scopes
- Testing in small sections is useful because the requirements are
more narrowly scoped and because any observed failures must be due to
defects in that section.
- Unit testing: Test one function, method, or class at a time.
Try to isolate that element from the rest of the product. Each
test is very simple. Assigning blame is obvious. Can be started
very early in development.
- Integration testing: Test specific combinations of components.
Other components may need to be replaced with stubs or drivers.
Test cases focus on interactions between components. Can be started
- System testing: Systematic testing of the entire system. Test
cases get larger and more complex. Harder to assign blame. Can
only be started after entire system is implemented.
- Staging: System testing in something very close to the system's
intended operating environment. Usually done with actual user
data. Can only be done just before deployment.
QA Plans > QA Activities > Testing specific quality goals
- Remember that the goal of QA is to assure that we meet our
quality goals. A good way to do that is to address a specific
- Functional testing: Verify that the system produces the correct
result. Can be done with any strategy, at any scope.
- Correctness: gives the right result for valid input
- Robustness: gracefully handles invalid input
- Accuracy: correct results are mathematically precise
- Compatibility: file formats, network protocols, browsers,
operating system versions, etc.
- Performance and scalability testing: Verify that the system will
perform well in heavy usage. Usually done with automated,
- Load testing: time to complete operations under heavy usage load
- Stress testing: gracefully handles excessive usage load
- Volume testing: performance with very large datasets
- Longevity testing: servers should continue to satisfy long sequences of requests
- Usability testing: Verify that the system will be usable by
- Understandability: users can understand how to use the system
- Learnability: the UI gives clues to explain unfamiliar items
- Efficiency of use: users can get their work done without too many steps
- UI safety: common human errors have limited negative impact
- Security testing
- Physical, network, operating system, application
- Encrypted communications
- Authentication: you are who you say you are
- Authorization: limited access, limited actions
- Malicious inputs
- Denial of service
- Operability testing: Verify that use cases for the system administrator work.
- Upgrade software, migrate data to new formats
- Recovery for system crashes or other errors
- Auditability: system keeps records of events for later review