In this posting we are going to review how teams use session-based test management techniques with our test management tool TestRail. With more and more teams switching to agile development practices, adopting continuous integration and deployment, and with ever and fast changing requirements, the role of testers and the expectation of testing teams evolve as well. Teams often don’t have the luxury anymore to carefully plan test strategies and tests for all new features in advance.
For many projects and development efforts, new features and improvements get released to customers as quickly as possible, leaving testing teams in the difficult position to quickly assess and review new builds and features as they get developed. One way to improve test coverage is to invest more resources in test automation. Another useful technique we introduced in a previous posting to overcome these challenges are staged releases.
But you can only automate so much and especially for new features, automated test coverage is often low. In addition to traditional test planning and manual testing, and in addition to test automation efforts, many teams use exploratory testing to quickly verify new functionality. Exploratory testing is an approach that enables test design and test execution at the same time. With TestRail’s recent update with its focus on exploratory testing features, we are going to review how to take advantage of this and implement session-based testing with TestRail.
Session-Based Test Management
Exploratory testing is unscripted testing, which means that you start testing and exploring the functionality of the system under test without planning your test cases in advance. This works great in combination with traditional test cases and regression testing, as well as with automated testing. But the challenge is, especially if you are working in a larger team, that you often still need a way to plan and structure your exploratory tests.
At the same time, you might have customers or managers who would like to make sure that all important feature areas have been covered. You might also want to track metrics, make it easy to assign tests to different testers and have a good approach to learn from your testing with formal debriefs.
This is where session-based test management (SBTM) comes into play. SBTM helps facilitate exploratory testing for software projects and at the same time ensures accountability, provides structure and makes it easier to manage test sessions for larger teams. With TestRail’s new templating features, our default exploratory session template and our unique & rapid FastTrack testing interface, teams can take advantage of and combine SBTM with other techniques all in one central place.
Planning Sessions in TestRail
With TestRail’s new exploratory test templates, it’s easy to plan your test sessions. You simply create a new test, select the exploratory session template and fill in your session details. By default, TestRail comes with fields to enter your Mission and Goals for a test session. But you can also freely define your own fields and add more custom fields under Administration > Customizations. This can be especially useful if you want to include additional information for your test session, such as the test environment or specific resources testers can use.
Now, TestRail provides a lot of features to group tests in sections, so you can easily group related sessions together and build a hierarchy of test sessions for your project. This way you can easily structure your exploratory tests and test sessions even for large and complex projects. And because exploratory sessions are just normal tests in TestRail, you can also easily reuse them for new releases or execute multiple sessions in parallel for different platforms to scale SBTM for large projects.
Plan test sessions with mission and goal descriptions in TestRail
Session Execution, Defects and Notes
To execute your test sessions and record notes, link defects and store screenshots as you review the system under test and functionality, you simply start a test run in TestRail. You can easily select and include multiple sessions and assign them to different testers, or directly plan your own upcoming sessions for the entire iteration. Because exploratory sessions are just regular tests in TestRail, you can also mix your test sessions with regular tests and test automation all in the same test run.
As you explore the system under test and discover issues, identify defects or notice things that could be improved, you can quickly add notes to your tests, add screenshots or log failures and push bug reports to a linked issue tracker such as JIRA, Redmine, FogBugz and many more. TestRail makes it super easy to add notes, screenshots and defects, so you can go through your system under test rapidly without slowing down.
You can fully focus on testing and spend less time managing your test session and taking notes. TestRail also makes it easy to record test times and mark the session as completed, either with a regular test status or by defining a custom status under Administration > Integration (such as Completed).
Easily record comments, screenshots, bug reports and results for session
Debriefs, Reporting and Metrics
An important part of session-based test management is learning from conducted test sessions and making testing accountable. To learn from your tests, doing a debrief of your test sessions together with another team member or stakeholder is a great way to improve future sessions, ensure you covered all important aspects of the system under tests and to review the effectiveness of your tests.
As TestRail keeps a transparent chronological history of all notes, screenshots and defects reported, you can easily review all your test sessions in a central place. TestRail also makes it easy to archive all your test runs for future reviews so you can always go back and see past session details. During debriefs it also makes sense to review which sessions you want to keep for future test iterations or to make adjustments to the session’s Mission and Goals descriptions.
Often times you also want to rerun sessions for different platforms or configurations, or reuse sessions for updated builds and releases in the future. TestRail’s test and run concepts make this very easy. Another important aspect of SBTM is having metrics such as session numbers, defects found and test times for your team to review and optimize. Fortunately, with TestRail’s rich reporting features, you automatically benefit from all the included reports and reporting templates for your session-based tests as well.
Live reports with defect statistics and other important project metrics
Every project requires a slightly different testing approach and each team has their own style and techniques they use based on their experience, skill level and available resources. It’s important to select the best approach based on the context. For many teams, combining manual testing, automated tests, exploratory tests and other techniques such as staged releases can be a great mix to meet the ever increasing demand customers have for software quality.
