Author: Satı BAKANOĞLU, System Test Specialist – Licensing Project Group
There are three main high level test documentation, which I came across all through my career in software testing. These documents are:
- Test Policy
- Test Strategy
- Test Plan
I believe that it would be useful to understand the purpose and the content of these documents, in order to both increase the quality of the work that we do and contribute to our field of experience.
1. TEST POLICY:
Test policy is a document described at the organization level and gives the organizational insight for the test activities.
It is determined by the senior management of the organization and defines the test principles that the organization has adopt. The test policy is very high-level document and at the top of the test documentation structure.
Organizations may prefer to publish their test policy in a sentence, as well as a separate document. Also they may use this policy in both development and maintenance projects.
The test policy shall describe the followings:
- Clear answer to the question of “What does testing means for the organization”
- Test objectives that the organization have
- The definition of the testing process used by the organization to increase the quality of the software developed
- How the organization will measure the effectiveness and efficiency of the test while achieving goals
- How the organization will improve its test processes
2. TEST STRATEGY:
Test strategy document is prepared at the program level and includes general test strategy, management principles, processes and approaches for the tests to be performed for a software in detail.
The test strategy document is also a high level document and is usually written by the test manager and the project manager in the top level organization. It is generally prepared in large scale projects and does not need much updating. In small scale projects, test strategies and test approach may be included in the test plan, and also the test strategy document may not be written separately.
Test approach and test activities included in the test strategy document must be consistent with the test policies of the organization.
The test strategy document may applicable for a program / system that contains multiple projects and describes;
- Objective / scope of testing
- In-scope / out of scope items for testing
- Test levels (Unit, System, Integration, System Integration)
- Test types ( Functional / Non-Functional)
- Entry / Exit / Stop / Resumption Criteria for testing (for different levels / phases)
- Risks to be addressed
- Test environment
- Test case design methodology
- Test methodology (Top-down / bottom-up / risk based)
- Test control and reporting
- Test automation approach
- Test tools to be used
- Defect management approach
- Defect classification
- Retesting & regression approach
3. TEST PLAN:
Test plan is a document prepared at the project level. In general it defines work products to be tested, how they will be tested (test cases) and test type distribution among testers. Test plan also includes test environment and test tools to be used during the project, the persons responsible for the tests and their responsibilities, test levels and test types, test schedule planned for test runs, and the principles of management and reporting of errors / bugs.
Test plan is usually prepared by the test manager or test leader in the test organization and shared with the entire team in the project. It is a living document throughout the project and should be kept under revision control as it’s updated.
The information in the test plan document must be consistent with the organization’s test policy and test strategy.
The test plan may describe the followings:
- All test strategies specific to the project
- Test estimations & test schedule
- Test organization / roles / responsibilities
- Test deliverables
- Test reporting principles
IEEE Std 829 (IEEE Standard for Software Test Documentation) gives a “Test Plan Template” that would be useful for those who will prepare a test plan. According to this standard, test plans shall cover,
- Test Plan Identifier (name, number, revision number of the document)
- Test Items
- Software Risk Issues
- Features to be Tested
- Features not to Tested
- Item Pass / Fail Criteria
- Suspension Criteria and Resumption Requirements
- Test Deliverables
- Remaining Test Task
- Environmental Needs
- Staffing and training needs
- Planning Risks and Contingencies
The companies working in the field of software development and testing should also be handled the above mentioned test documentation in their quality management system. Thus, the company guarantees that the software developed will provide and maintain the quality in the best possible level. All employees involved in the software testing process know that a software will not be 100% error-free. However, the presence of these documents and setting testing criteria within them, e.g. exit / resumption criteria or requirements, are mandatory for a company to protect itself formally.