Now that TestRail 2.0 is available, we wanted to share some of our plans for future versions of our test management software. There are various things we plan to introduce over the coming months and in the next year and we have already communicated some of our plans in our forum and via email with some of you.
Before explaining some of the features and enhancements we plan to add to TestRail, let me first state the usual caveats regarding roadmaps: while the features mentioned below are some of the enhancements we currently plan to add to TestRail, our priorities might change and all plans laid out in this posting are subject to change. Additionally, features might not necessarily be added in the same order as listed below and we don’t currently plan to announce release dates or time frames for the enhancements in advance.
That said we understand that customers are curious about upcoming features and this posting should help you understand where we are going with TestRail. You can still help us define the scope of planned features by filling out our recently published TestRail survey.
Time Reporting & Forecasting Released with TestRail 2.1
TestRail already contains functionality to enter time estimates for test cases and to record actual execution times for tests. TestRail even comes with a handy test timer that automatically tracks execution times for you. Our plan is to release time and progress reports in TestRail’s next update to help you make sense of the entered data.
However, we will go beyond simply listing the recorded times and estimates: as tests are usually executed multiple times over the lifespan of a project, TestRail can use the historical execution times to generate forecasts for upcoming test runs, plans and milestones. We’ve developed a sophisticated algorithm to generate those forecasts based on entered estimates, historical execution data and averages. We expect that our forecasts will get very accurate over time as you enter more and more data. This will be a very unique feature of TestRail that will help you schedule and track your software tests.
Requirement & User Story Integration Released with TestRail 2.1
Many existing TestRail customers are already storing their requirements (or user stories) in their issue tracking tool, a Wiki, CMS or a dedicated requirement management tool. While we are also considering adding an optional built-in requirement management module to TestRail at some point, our focus for now is to better integrate TestRail with existing solutions.
This will be accomplished by adding a new built-in References field to test cases. The References field will work similar to how defects are linked to test results: you can enter the IDs of linked requirements/user stories and optionally link those IDs to external web-based systems. A coverage report for references is also planned but will likely be added later.
Improved Bug Tracker Integration Released with TestRail 2.2
The current integration with bug / defect / issue tracking tools served us very well so far and is flexible enough to work with almost all web-based tools. However, a deeper integration with bug trackers is desirable to make it easier to report bugs and to look up the status of issues directly from within TestRail. Improving this has a high priority for us and most of the planned functionality has already been designed.
Our plan is to keep the current linked-based integration and to allow TestRail administrators to setup an additional integration script that allows users to report bugs from within TestRail. A new Push link next to the Defects field will allow users to submit bug reports from TestRail to the bug tracker, using the bug tracker’s web service and APIs. It will also be possible to look up the status of issues in TestRail and we are considering additional integration points as well. The integration scripts will be customizable and we also plan to ship ready-to-use scripts for popular bug tracking tools with TestRail.
Customization
Making TestRail more flexible and customizable is not a single milestone or task that we will implement and release with a specific new version. However, since we plan quite few enhancements to make TestRail even more flexible, we wanted to mention this in our roadmap. Some of the customization options we consider (or that are already planned) include a test case filter to make it easier to select test cases for runs, customizable columns for grids, better sorting and filtering options and customizable overview pages.
Better Reporting
While TestRail already provides many useful metrics, reports and charts in its user interface that are more accessible than the reports most other tools provide, we understand that better reporting capabilities would be a very useful addition. Our plan is to add additional reports to TestRail, such as coverage, matrix and defect reports, and make them accessible from a central place in TestRail (possibly on a dedicated Reports tab). We will continue providing useful metrics directly in TestRail’s user interface and reports will also be accessible on the entity pages directly. However, providing more reports that are easier to customize and having all reports accessible from a dedicated page would make the life of team leads a bit easier.
Improved Automated Testing Support
If you want to integrate TestRail with automated testing tools, you can already do this today with our HTTP-based Mini API. If you compare this integration option with other test management tools, you will find that our API is as powerful as or even more powerful than most other systems. However, we believe that we can do better.
There are currently two ways we are considering to improve our automated testing support (and we would certainly love to receive your feedback about this). The first is to allow developers and testers to submit test results from automated unit and GUI tests to TestRail in batch mode. This would allow you to directly upload your test results from your automated software builds (or scheduled test executions) to TestRail all at once. TestRail would automatically analyze the test runs and make the reports available in TestRail’s user interface, separately from manual tests.
The second option is to allow testers to trigger automated tests (and possible store test scripts or data in TestRail) from test pages. It is likely that we will support both workflows at some point, as the feedback we received indicates that there isn’t one best approach to handle automated tests and that each team has its own preferences.
Comprehensive API
As mentioned previously, TestRail already has an API to submit test results to TestRail. Additionally, you can also directly access TestRail’s database to manipulate data, generate reports and run queries. However, a comprehensive web service API would be a useful addition to make it easier to integrate other systems with TestRail (without opening up TestRail’s database). Additionally, a more advanced API will come in handy when/if we provide a hosted edition of TestRail, as direct database access will not be possible with the hosted edition. It is likely that we will implement this as a REST-like API with XML/JSON result sets, making it easy to access the API from many platforms and programming tools.
Closing Notes
There are also other unique ideas we are considering adding to TestRail that we don’t want to disclose at this point (competitors need to come up with their own ideas, right?). That said, we will certainly also see additional enhancements for Agile teams in the future and it’s also likely that we will offer a hosted version of TestRail at some point. Feel free to email us if you have any questions, suggestions or comments about this roadmap or TestRail’s future.
