We’re pleased to announce that TestRail 6.7 has been released to our hosted TestRail Cloud customers, with various changes and enhancements designed to improve performance and scalability for all TestRail cloud customers.
In brief:
- We’ve introduced a new Data Storage Policy for TestRail Cloud customers that will take effect on February 1st, 2021, along with new admin features to help monitor and manage your storage usage.
- We’ve made some small changes to some data export and reporting settings to reduce unintended impact on performance.
- We’ve added pagination to certain pages of the web app and all bulk API methods to more gracefully serve customers with large amounts of test cases and historical testing artifacts (results, runs, milestones, etc). Customers who currently use any bulk TestRail endpoints must switch to the new paginated bulk API endpoints by February 26th, 2021 to avoid any interruptions in service.
- We’ve added a 90-day password expiration policy for users that have a Gurock Customer Portal account for billing and license management. This does not affect any users of the TestRail app.
Data Storage Changes
The biggest change for this release is the introduction of some adjustments to how data storage is currently reported in hosted TestRail Cloud instances, and what reporting on data storage means for customers going forward.
Historically, we haven’t reported on data storage for our cloud instances at all! And generally, that hasn’t been too much of a problem, since for most customers our out-of-the-box storage plan has been perfectly adequate for both their (and our) needs.
To be completely clear:
- Our hosted TestRail customers need to be able to store their testing data (test cases, tests, runs, plans, etc., along with any associated attachments and generated reports).
- Our hosted TestRail customers also need to be able to export that data out of TestRail, should they wish to transfer it, or back it up locally.
We (Gurock) should be able to back-up hosted instances on a daily basis so we can restore them to a prior state if needed.
Unfortunately, some (less than 1%) of hosted TestRail Cloud customer instances have grown too large for some of these operations to be carried out safely, successfully, and without impacting other customers hosted across our cloud infrastructure. While we continue to make performance improvements in the platform itself, we had to think of a way to change our current storage model and policy, such that it provides all our customers with the best possible TestRail experience.
As a result, in TestRail 6.7 we have introduced a data storage policy and a number of features in the TestRail app to help your team monitor and manage your storage.
Data Storage Policy
For more information about our Data Storage policy you can refer to the full policy posted here, but here are the highlights:
Starting February 1, 2021:
- Professional TestRail customers will get 50GB data storage included in their cloud subscription.
- Enterprise TestRail customers will get 500GB data storage included in their cloud subscription.
As part of the improvements included in this release, we have migrated TestRail Enterprise instances to a segregated, more resource-intensive cluster to ensure high performance even with larger storage amounts being used.
We recognize that for some of our Professional customers, 50GB may not provide enough data storage space for their needs. So we have introduced some further billable storage options, billed at $10 per month for 25GB increments up to a maximum of 200GB.
Basically what this means is that, if you go over your Professional allowance, we will charge you the additional $10 per month for each 25GB increment, based on the peak storage amount reached during the billing period – up to the maximum allowable 200GB. This charge will be added to your regular monthly bill. If you reach 200GB, you have the options to:
- Reduce your data storage footprint by carrying out some remedial actions (refer to our TestRail Data Management documentation).
- Upgrade your subscription to Enterprise, thereby bumping your allowance to 500GB.
- Export your data to an on-premise installation of TestRail.
For both Professional and Enterprise tiers of TestRail, if you reach the maximum allowable storage capacity (200GB and 500GB respectively), we will notify you about the overage and help find a solution to reduce your storage footprint. If you do not take any actions to reduce your storage, some features may be restricted so as to prevent their impacting performance in our cloud infrastructure:
- Data can’t be exported from the administration console (CSV & XML test case exports are allowed still).
- Additional custom fields can’t be created.
- Additional attachments can’t be uploaded.
Hosted customers will be able to see what their current data storage looks like in the TestRail Administration > Overview console:
Administrators will also be able to review their current data storage footprint in the new Data Management area:
You’ll also find the enhanced Data Export feature there too, so let’s talk about that.
Data Export Changes
We’ve made a few changes to the export function in TestRail. As of 6.7, when scheduling an export you’ll need to specify up-front which version of SQL you want to use: MySQL or MSSQL.
Additionally, unless you check the boxes to include either reports, attachments, or both, they’ll be excluded from the export by default.
We’ve made these changes for the same reasons as implementing the data storage policy. Carrying out those exports and including all of the information by default resulted in performance risk across our cloud infrastructure. Reducing the size of those exports by keeping them to a single SQL version and only including explicitly requested data helps us to reduce the risk of performance degradation for all our hosted customers.
Pagination & API Changes
Pagination is another big feature for our scalability release, affecting both the TestRail UI and the TestRail API.
Pagination in the UI
For the UI, we’ve implemented pagination on all of the pages that could be impacted by attempting to load a large number of test cases, tests, runs, and other TestRail entities by default. The full list of pages where pagination has been implemented is here:
- /runs/view/
- /tests/view/
- /cases/results/
- /runs/overview/
- /suites/runs/
- /suites/overview/
- /plans/view/
TestRail administrators can control the pagination level across those pages using the Pagination Limit settings in the Administration > User Interface console, shown below.
Pagination in the API
We’ve also introduced pagination for bulk requests in the API. In the future, when you make a bulk request using a TestRail API endpoint such as get_cases, you’ll see that the response structure has changed so that it returns an object with additional pagination fields and an array of up to 250 entities.
All bulk API endpoints will contain the new structure and the pagination links. If your request results in a response that exceeds the maximum number of objects allowed, you will receive the first 250 of them and a link to the next set of results in the response body.
{"offset": 0,
"limit": 250,
"size": 250,
"_links":{
"next": "/api/v2/get_cases/1&limit=250&offset=250",
"prev": null
},
"cases":[
{ "id": 1, "title": "..", .. },
{ "id": 2, "title": "..", .. },
..
]
}
You can apply an offset to your request, or further limit the number of entities returned by adding offset and limit parameters to the request URL:
GET index.php?/api/v2/get_cases/:project_id &suite_id=:suite_id limit=:limit &offset=:offset
The full list of bulk API endpoints that include this change are below:
- get_cases
- get_runs
- get_results
- get_tests
- get_results_for_case
- get_results_for_run
- get_plans
- get_projects
- get_sections
- get_milestones
- get_history_for_case
- get_attachments_for_case
- get_attachments_for_run
- get_attachments_for_plan
6.7 API Beta Period
We recognize that it may take some time to adjust to these changes. We will have therefore released them in beta, so that API users have sufficient opportunity to make any necessary changes to code and integrations which leverage the endpoints.
If you currently use one of the bulk API methods enumerated above, you must switch to using the new paginated API method described above before February 26, 2021.
In our TestRail 6.7 release, the API will use the old API requests/responses for all the endpoints above.
If you want to try out the new API, you can do so by adding the following header and parameter to your API requests:
--header 'x-api-ident: beta'
After the beta period has finished, the updated endpoints will be used for all responses.
Further information about the changes can be found in the TestRail API documentation.
Report Changes
The 6.7 release also contains some minor changes and a fix to TestRail reports.
For performance reasons, we have removed the capability to select “all” for the specific report dimension. For example, when running the Comparison for Cases report, you won’t see the “All Test Runs” radio button any longer.
We’ve made this change for similar reasons as the other 6.7 features. On a large hosted instance of TestRail with a great number of runs, generating a report for all runs by default was a somewhat risky approach. Executing that report could potentially take a long time, and have a performance impact on the hosted instance while it was being generated. We have therefore removed “all” as a default option for any reports which used it.
Of course, if you really do want to report on “all” for a specific dimension, you can certainly do so still, by setting an appropriate filter.
Oh, and we also fixed a bug to prevent reports being endlessly rerun in the event of a failure. Because, you know, performance.
Updates to the TestRail Customer Portal
Finally, to improve security for all TestRail customers, we have introduced a new password 90-day expiration policy for the Gurock Customer Portal accessible at https://www.gurock.com/portal/. This password expiration policy will not affect any app users; it only applies to users named as primary, technical, or billing contacts for TestRail cloud subscriptions or server licenses in reference to their customer portal account.
Getting TestRail 6.7
You can start a 14-day free trial of TestRail here (cloud or server): https://www.gurock.com/testrail/trial/
If you want to create a subscription for TestRail Cloud, you can do so from within TestRail via Administration > Subscription. Or, if you want to order TestRail Server licenses you can do so from our website here: https://secure.gurock.com/customers/shop/annual/purchase/
Registered customers can download the full version from our customer portal: https://www.gurock.com/portal/
TestRail Enterprise
If you’d like to learn more about increasing your storage limit or enabling project-level administration, single sign-on, or any of the other features on TestRail Enterprise, please email us for a trial or quote via [email protected], or you can use the contact form here: https://secure.gurock.com/customers/support/.
Not sure which TestRail plan you’re on? Reach out and we’d be happy to help.
Updating to TestRail 6.7
TestRail Cloud
TestRail Cloud instances are automatically updated to the latest version. You can check your version via the TestRail Help > About TestRail menu item.
TestRail Server
If you are using the on-premise version of TestRail, you can update to the new version as usual by installing it over your existing TestRail installation (there’s no need to uninstall your existing installation). The database upgrade wizard is automatically started when you access TestRail with your web browser. Please see the update instructions for details: https://docs.gurock.com/testrail-admin/installation-upgrading/
If you’re using Docker, please refer to the instructions here: https://docs.gurock.com/testrail-admin/installation-docker
Please ensure you make a backup of your current on-premise installation before upgrading to the new version.
