In our new TestRail Highlight blog series we will take an in-depth look at select features of our test management software TestRail. As software projects evolve, every testing team needs to think about how they will handle changing requirements and test cases for their projects and releases over time. Today we will take a look at TestRail’s unique features that help help teams archive test runs, track the history of test case changes and easily manage baselines for different project release branches.
Not every team needs to use the full capabilities of these features. One of our main design goals for TestRail is to make the application as easy to use as possible, while at the same time providing all advanced features large teams need to grow with TestRail. For example, basically all teams using TestRail will benefit from TestRail’s test case history functionality or the ability to archive testing efforts. But not all (or even most!) teams need to use TestRail’s advanced baseline project mode. Please see below for an overview of TestRail’s rich project and test case versioning support!
Archiving and Closing Test Runs/Plans
In addition to managing and organizing all current testing efforts, many teams adopt a test management tool so they can easily store, track and review past test results. TestRail makes it very easy to start many test runs against your test cases over time (e.g. for different iterations and project releases), and even start test runs in parallel (e.g. to test different platforms and configurations at the same time). This makes it easy to reuse the same case library without duplicating or synchronizing any test cases.
But if you are changing and improving your test cases for new project versions, wouldn’t old test runs show the wrong test case details? Isn’t it critical that old test runs show the exact test case details they were tested against? Enter TestRail’s Close Run/Plan feature. TestRail provides a unique feature to easily archive and close test runs and test plans. When you close a test run or plan, TestRail actually archives and copies all related test case details behind the scenes. So if you move, update or even delete test cases in the future, closed test runs wouldn’t be affected by this and would still always show the exact test case details your team tested against.
TestRail also prevents testers from making any additional changes to a closed test run and its results, so you can always review your previous test results and be sure that the archived runs weren’t modified. While at first a seemingly simple feature, TestRail’s Close Run/Plan option requires a lot of work behind the scenes to accomplish our goal of making closed test runs immutable. This functionality is critical for basically any testing team and surprisingly few test management tools actually offer similar features that come close to TestRail’s implementation.
Close test runs & plans for accurate test case and result archives
Full Test Case Change History
TestRail makes it easy to update your test cases at any time, so you can directly improve and change your test cases during testing or when you review your case library. This is especially helpful when your application and development is under constant change and if you need to adopt to changed requirements quickly. Test cases changes are also automatically reflected in all active test runs. So especially teams that use exploratory testing and other agile testing techniques benefit from live updating and improving test cases during testing.
With all the changes teams make to test cases over time, it’s often helpful to know what changes were made, who made those changes and when a test case was updated. TestRail keeps a detailed log of all changes that were made to a test case and you can easily review this log. When you open a test case in TestRail, simply switch to the History tab from the sidebar. Not only will you be able to see when a test case was updated and who made the changes, but you will also see a detailed diff of all changed test case attributes. This also makes it easy to revert any changes by copying previous test case details to the latest version.
Full test case history to track test changes by person, date and content
Project Suite Modes and Baseline Support
When we originally introduced TestRail 4.0 about 18 months ago, we added new suite modes to manage your test cases. In earlier TestRail versions we enabled test suites by default to organize your test cases in top level test suites and sections. As we invested a lot of resources over the years to make test suites much more scalable and as we introduced new view modes, we decided to default to a single test case repository per project. Teams can still enable and use test suites, but for most projects using a single case repository per project and organizing test cases via sections and sub sections works much better.
But this is not the only option we introduced at that time. Lesser known to many TestRail users, we also added new baseline support to projects. Baselines allow you to create multiple branches and test case copies within a project. Internally a baseline is similar to a test suite, and it includes all sections and test cases of the original baseline or master branch it was copied from. You can switch a project to baseline mode under Administration > Projects > edit a project.
Should your team use and enable baseline support for your projects? Probably not! Teams should only use baselines in a specific situation: if you have to maintain multiple major project versions for a long time in parallel. That is, your team needs to maintain and release multiple main branches (e.g. 1.x and 2.x) in parallel for many months or years, and the test case details will be quite different for each baseline over time, as each version will need different test cases. We designed baseline support specifically for this scenario and it’s a great way to manage multiple branches and versions of a project’s test case repository for multiple parallel development efforts.
Using baselines to maintain multiple parallel test case repository branches
The above mentioned features make it easy to manage, track and review different test case versions and changes over time. Not every team will need all of the above features though. Especially the baseline mode should be limited to projects where this is really required. But TestRail offers advanced versioning and history tracking features for all kinds of scenarios and configurations.
In addition to the mentioned tools to manage your test case versions, TestRail also comes with various helpful options to track your releases, iterations and sprints via milestones and test plans. Make sure to try TestRail free for 30 days to improve your testing efforts if you aren’t using it yet, and check out TestRail’s great versioning support.
