In software development, a test plan defines your testing team’s test strategy, goals, and scope, which ultimately work together to ensure that all your software components are tested sufficiently before a release.
Follow these six steps to create an efficient test plan:
- Define the release scope
- Schedule timelines
- Define test objectives
- Determine test deliverables
- Design the test strategy
- Plan test environment and test data
How to create a test plan
1. Define the release scope
Before any test activity occurs, it’s important to define the scope of testing for your release. This means defining the features or functions that need to be included in the release, considering any constraints and dependencies that can affect the release, and determining what type of release it is.
Examples of questions to ask when defining the release scope include:
- Are there new features being released in this version?
- What are the risk areas?
- Are there any particularly sticky areas where you’ve seen regressions in the past?
- What type of release is it? Is this a maintenance release that includes bug fixes? Is this a minor feature release? Is this a major feature release?
- What does being “done” actually look like for your team?
For example, what information would you require if your organization has just launched a new e-commerce site and wants to test it before it launches?
Whether it’s talking with the developers to understand the scope of the project, or working with a product manager to walkthrough new functionalities and user flow, defining the scope ensures that accurate information is being shared and that there is a common understanding of the product’s goals, expectations, and features.
2. Schedule timelines
Specify release deadlines to help you decide your testing time and routine. Here are some pointers for determining timelines:
- Consult your project manager to understand the current release timeline.
- Look at past release times and schedules.
- Consider extraneous elements: Does the release need to coincide with outside variables, such as conferences or events? Factor those into your release date prediction.
- Consider the timeframes for development: Your development team might have a set schedule for finishing development work. Make sure you comprehend that timeframe so you can adjust the testing schedule.
- Add some extra wiggle room: It’s common to encounter unexpected delays. Including extra time for unforeseen events can help you stick to your plan.
- Review and update the schedule frequently to ensure the test timetable is attainable.
3. Define test objectives
A test objective is a reason or purpose for designing and executing a test. These objectives ultimately help guide and define the scope of testing activities.
Examples of general test objectives include:
- Identifying and reporting defects
- Testing new features
- A certain level of test coverage
Examples of objectives for specific types of testing include:
- Functional testing objectives: Ensure the software works as it should. Examples of goals for this objective include: Validating user workflows, data processing, and verifying input/output parameters.
- Performance testing objectives: Ensure the software is efficient and can handle various loads. Examples of goals for this objective include: Verifying software reaction time, throughput, and scalability.
- Security testing objectives: Uncover program security flaws. Examples of goals for this objective include: Verifying authentication and authorization features and identifying potential threats.
- Usability testing objectives: Concentrate on ease of use and user experience. Examples of goals for this objective include: Validating software accessibility, verifying user flow, and identifying user-related issues.
Measure testing with the right metrics
Metrics assess the overall quality of a release, the progress of your testing, and the effectiveness of your testing (for a particular test cycle or the entirety of your testing).
They provide visibility into your testing process and overall product quality, ultimately helping your team decide if your release is ready to ship. Here are some metric formulas you might consider:
- Defect Density = Defect count/size of the release (lines of code)
Example: If your software has 150 defects and 15,000 lines of code, its defect density is 0.01 defects per line of code.
Test Coverage
- Test Coverage = (Total number of requirements mapped to test cases / Total number of requirements) x 100.
Defect Detection Efficiency (DDE)
- DDE = The percentage of defects detected during a phase / Total number of defects
Time to Market
- TTM = The time it takes for your company to go from idea to product launch
4. Determine test deliverables
Test deliverables are the products of testing that help track testing progress. Deliverables should meet your project’s and client’s needs, be identified early enough to be included in the test plan, and be scheduled accordingly. There are different test deliverables at every phase of the software development lifecycle. Here are important deliverables to focus on before, during, and after testing:
Before testing
- Test plan document: The scope, objectives, and approach of the testing endeavor are all outlined in the test plan.
- Test suite: Test cases illustrate how to run a test, including input data, expected output, and pass/fail criteria.
- Test design and environment specifications: The test environment outlines the hardware and software configurations used for testing.
During testing
- Test log: The test log records each test case’s results, including issues and resolutions.
- Defect report: A defect report lists testing issues by severity, priority, and reproducibility.
- Test data: According to the International Software Testing Qualifications Board (ISTQB), test data is data created or selected to satisfy the execution preconditions and input content required to execute one or more test cases.
- Test summary report: The test summary report lists the number of tests run, passed, and failed, as well as open defects.
After testing
- Test completion report: Covers the testing scope, product quality, and lessons discovered.
- User acceptance test (UAT) report: Points to any issues found and fixed.
- Release notes: List information about what the release includes. Examples include any new features for development, advancements, or fixes.
A test plan’s content and structure differ depending on its context. Although there isn’t one cookie-cutter way to write a test plan, following best practices for test plan development can help companies deliver high-quality software.
TestRail is test plan software designed to make it easy to follow best practices for test plan development. In TestRail, you can enter test cases with preconditions, test instructions, expected results, priorities, and effort estimates.
Image: Organize and structure reusable test cases in folders, create agile test plans, and track test execution progress in TestRail.
This level of flexibility and visibility into your testing process makes TestRail an easy fit into any organization’s test plan — Try TestRail for free to see how it can help with your test planning.
5. Design the test strategy
Test strategy helps determine test cost, test effort, and which features will be in-scope (planned to be tested) versus out-of-scope (not planned to be tested).
Identify testing types
It is critical to identify when to perform what type of testing, what should be tested manually vs. automated, the scope of automated tests, how much work will be required to create new test cases, and who will be doing that work.
Depending on several factors, there may be various types of testing to include in your test plan.
Examples of factors to consider when choosing the right testing type to perform include:
- Test objectives
- Your project’s feature requirements
- The complexity of your product
- Your team’s experience levels
- Regulatory requirements
- Time and budget
Here are commonly used types of testing to consider including in your test plan:
| Manual Testing | Automated Testing | Other |
| •Smoke testing •Exploratory testing •Usability testing of new features | •Unit testing •Regression testing for existing features •Integration testing | •Performance testing •Security testing •Accessibility testing |
Document risks and issues
It’s essential to document risks that may occur during testing and the effect of these risks. Risks can include:
- Strict deadlines
- Insufficient or inaccurate budget estimates
- Poor management
- Problems with the code
- Changes in the business environment
- Limited resources for testing
- Unexpected delays during testing
Document test logistics
Test logistics should answer the “Who, what, where, when, and how.” Documenting test logistics ensures that all human and system-related testing resources are available. For example, it may be important that your team identifies who is available to do testing and who will support them if needed during testing. Moreover, when resource planning, it can be helpful to identify alternative resources or build slack into your plan to ensure your project gets completed.
Establish test criteria
Test Criteria is a standard that regulates all activities within a testing project. The two main types of test criteria include suspension and exit criteria.
- Suspension Criteria: Establishes the conditions for suspending all tests.
- Exit Criteria: Exit criteria are established items or goals to complete that define the end of a test phase. The exit criteria of a test are the predetermined results that must be achieved to move on to the next testing phase. For example, 92% of all critical test cases must pass before a feature can be deemed suitable for release to your customers.
6. Plan the test environment and test data
Planning a test environment guarantees precise and robust testing. The test environment includes hardware, software, and network configurations for software testing. Follow these procedures to set up the test environment:
- Determine your hardware and program requirements: Select test environment devices and software, including operating systems, browsers, databases, and testing tools.
- Install the required software: Once prerequisites are established, install the necessary tools on the test environment. This may require setting up a separate server to host the application and installing a database management system or other tools.
- Configure the network: Make sure that firewall protocols, IP addresses, and DNS settings, among other network configurations, are identical between the test and production environments.
- Create the test data: Prepare the test material for the application’s testing. Test data can be created manually with data from the production environment, retrieved from an existing production environment and database, or, created via automated Data Generation Tools.
- Access the builds: Ensure that the builds that the testers will be testing are accessible. One example is setting up a file-sharing or version control system to allow testers access to the most current builds.
- Verify the test environment: After setting it up, check that your test environment fulfills the requirements.
One-page test plan template
| Test Plan Title Prepared by: John Doe |
| 1. Introduction •Executive summary (This should be kept brief) |
| 2. Testing Resources •Tester’s name and role |
| 3. Scope of Testing •In scope: Modules that are to be tested •Out of scope: Modules that are not to be tested |
| 4. Testing approaches •Testing approach and methodology •Types of testing to be performed (e.g., functional, performance, security, usability) |
| 5. Test Schedule •Timeline for each testing phase |
| 6. Risks & Issues •Risks associated with the testing process •Mitigation strategies for identified risks |
If your test plan doesn’t fit onto one page, don’t worry. The intention is to minimize extraneous information and capture the necessary information that your stakeholders and testers need to execute the plan.
Test planning in a test management tool
Test management tools can help your team with their test planning efforts. Whether your goal is to be more agile or accelerate test planning, there are tactical approaches to making a comprehensive and agile test plan with a test management tool like TestRail.
- Milestones
In TestRail, milestones are containers for aggregating test artifacts and tracking different testing activities related to the same outcome. For example, if you create a milestone in TestRail to keep track of a release and connect all of the related test runs and test plans, then you can see all of the different activities you will do for that release in one location.
Image: Manage all your milestones and ongoing test projects in TestRail.
Within TestRail’s milestones, the description field provides a great place to build a one-page test plan. By having all of your test plan information connected to the milestone from the beginning, you can constantly refer back to it as you are planning and ask yourself questions such as, “What was in scope?” “What was out of scope?” “What are we testing or not testing?” and “Are we going to do automated testing in this milestone?”
Milestones give you an efficient way to quickly refer back to your test plan information from within your actual test management tool.
- Test case priority and type
Test cases define what you will test beforehand; essentially, they outline what you will do before you do it. In TestRail, you can organize test cases based on many hierarchies. This is a key way to start building your test plan.
For example, if you’re developing an advanced messaging app, the highest risk area to your app is that it has to be able to be installed and run. In this example, you might start with that as your smoke test and then do more in-depth functional or exploratory testing.
By capturing test case priority and what type of testing approach you plan to take with a particular test case ahead of time, you are starting to think about planning that test case in practice.
Image: Organize your TestRail test case repository based on priority.
- Test Reports
Using a test management tool like TestRail for your test execution is beneficial because you can use real-time reports. For example, if you built your plan out in TestRail and you’re executing around that plan, a report like TestRail’s milestone summary report is going to show you your initial test objectives, your initial one-page test plan, all the test runs and plans added within that milestone, the priority you assigned to them, and more. You can also download reports to share with your team or other stakeholders.
Image: The Milestone (Summary) report allows users to attach project criteria and quickly visualize a clean, shareable interpretation of milestone activity.
As projects become more complex, using spreadsheets as test plan templates can become unwieldy, and a more sophisticated approach is needed. Use a test management tool like TestRail to make your test plans as flexible as possible.
FAQS
Importance of test planning
- Ensures higher quality releases: A well-written test plan can help ensure that all key software components are thoroughly and methodically tested, lowering the risk of overlooked defects and bugs.
- Saves time and improves test efficiency: A roadmap of testing procedures and activities allows teams to prioritize tasks. A clear test strategy can also be reused, which helps save time and money during planning.
- Keeps stakeholders informed of the testing process as a whole.
- Guides QA engineers to meet requirements: Test plans provide QA engineers with direction for carrying out their testing tasks and ensure that testing procedures align with both technical and business criteria. This also means there is a lower probability of defects slipping through during testing.
Components of a test plan
| Components of a Test Plan | Description |
| Scope | •Defines the boundaries of the testing endeavor •Specifies the subject of the test •Specifies any features or functionalities to be tested |
| Out of Scope | •Describes the features/functionalities that are purposefully left out of the testing effort •Defines what is not being tested |
| Timeline | •Sets expectations for testing completion •Outlines the timetable for each testing phase, including milestones and deliverables |
| Resource Allocation/Roles and Responsibilities | •Describes the roles and responsibilities of team members involved in the testing effort •Defines resource allocation for each testing phase |
| Tools | •Describes the test equipment to be used (test management tools, automation tools, CI/CD tools…etc.) |
| Environment | •Defines the criteria for the test environment •Describes the hardware, software, and network configurations that make up the test environment |
| Deliverables | •Describes what you expect to come out of each testing phase (such as test reports, test results, and other relevant documents) |
| Defect Management | •Describes how to report, track, and manage bugs found during testing •Defines the severity levels of the bugs and how to fix them |
| Exit Criteria | •Defines the criteria for finishing each testing phase •Defines the criteria for accepting or rejecting the system under test |
A test plan’s content and structure will differ depending on the context in which it is used. For instance, in agile development, the test plan might need to be changed often to keep up with changing goals.
If you are using DevOps processes, the test plan may need to explain how testing will integrate with your development pipeline, what parts of your testing will be covered by existing automated tests, and what new tests you will try to automate during this test cycle.
