We’re excited to announce the general availability of TestRail 7.6 for cloud customers on the Early Access track.
We just made it even easier to complete your work in less time with the introduction of Test Parameterization for TestRail Enterprise customers, letting you leverage dynamic data with Test Parameterization for faster test case design and less maintenance. Users on all TestRail plans can now streamline workflows through an improved Jira integration that allows all customers to edit Jira issues directly from within TestRail and provides increased visibility into associated TestRail data from within Jira.
Please note: At this time, this release is available for cloud customers on the Early Access release track only. Click here to switch to Early Access and receive TestRail 7.6 now. Learn more >
Overview
Test Parameterization lets you leverage dynamic data to scale up test case design and test planning, helping you increase test coverage and reduce the amount of time you spend on test maintenance. By managing datasets and variables that can be reused across your test case repository, it’s easier than ever to ensure each of your test scenarios are executed accurately, yielding results that you can rely on. We also include a few new features in the Test Parameterization management interface that builds on our tradition at TestRail to make it the most intuitive test management solution available today.
New improvements to the TestRail integration with Jira helps teams using Jira save time and simplify their workflows by editing Jira cases directly from within TestRail and enjoying faster loading speeds in Jira. Teams using any requirements tracker can also increase traceability by associating Test Plans and Milestones with requirements.
Finally, teams using Jira can add visibility into testing across their organization by viewing them directly from within Jira – even for Jira users without a TestRail account.
Test Parameterization
Prior to TestRail 7.6, TestRail users were required to manually enter test case criteria into each and every test case. The core of the new functionality is the ability for testers to use centrally-managed reusable variable data for testing purposes. By referencing variables in test cases, you can reduce not only the amount of time spent writing test cases for various permutations of test criteria, but also greatly reduce the overall quantity of test cases in their TestRail instance – all with fewer chances of human error.
Administrators can assign add/modify/delete permissions to roles, ensuring the Test Parameterization data is being managed by the proper users. Testers with adequate permissions assigned to their roles will have a central area for managing Test Parameterization variables and datasets within their projects, allowing them to view and update data as needed so they can quickly and easily create test runs using their datasets.
Test Data and Variables
Within TestRail, the term “test data” refers to the general practice of reusing variables, datasets, and values specific to each project that can be called to quickly insert variables into test cases and to generate test runs based on existing datasets. As we’ll soon demonstrate, when a dataset is used by a test run, testers will see their test data appear in their tests, greatly increasing the flexibility of test cases and allowing for greater test coverage with fewer cases to manage.
Test Data Management
Each project in TestRail includes a dashboard dedicated to viewing and managing test data. You can access the dashboard from your lists of test runs, test suites, or test cases by clicking the View Test Data button.
The test data dashboard displays your project’s test data represented by a table that displays your existing datasets and variables. Note that test data is unique to each project.
- A Variable is a placeholder that you insert into your test cases. You can have up to 500 variables per dataset.
- A Dataset is a collection of values corresponding to each variable. There is always one default dataset and you can add up to 99 more datasets for a total of 100.
Datasets are NOT required to have a value for every variable. If a dataset’s value is blank, then any associated test cases use the value from the default dataset, if it exists.
The TestRail dashboard allows you to perform the following actions:
- Edit your test data – click Edit to open the edit mode for your test data.
- To add a new variable, type a new variable name into a blank row of the table’s Variable column.
- To add a new dataset, type a new dataset name into the header row of the table. You can then add new values to your new datasets.
- Click on the row number or column header to select the entire row or column. To delete the entire selected row or column, click delete the values.
- You must click Save to retain your changes.
IMPORTANT NOTE: Open test runs always retrieve the current value of the dataset and variable. Editing the values inside of your datasets affects any tests in open test runs. Closing a test run or test plan captures and preserves the dataset values at the time of closure, which creates a static record.
- Create test runs – click Create Test Runs to create test runs based on your data sets. This action opens the Create Test Runs dialog:
Select the dataset(s) you would like to use for your test run(s), and then click Continue. TestRail now allows you to select what you want to do with the selected dataset(s), as shown in the following dialog.
This dialog provides you with the following options for your new run(s):
- Create a test plan containing your test runs
- This option creates one test plan that contains a test run for each of the datasets selected. TestRail then opens the view for creating a new test plan with your new test runs included.
- Add your test runs to a current test plan
- This option creates a new test run for each of the datasets selected, but also allows you to add these test runs into an existing test plan. Choosing this option prompts you for the target test plan, and then adds these runs automatically to the selected plan.
- Create individual test runs
- This option creates a new, separate test run for each of the datasets selected. You can select test cases for these runs, but all of the test runs are created with the same selection of test cases.
IMPORTANT NOTE: Please keep in mind that it is not possible to add these options to a new or existing test run later.
- Import test data
Click Import CSV to import a CSV file of datasets and values.
Select your file and the appropriate file encoding, and then select the CSV delimiter to begin the import process.
TestRail displays a preview of the imported data. Verify that the preview looks accurate before importing the data. Any items appearing in green reflect an updated or new value.
IMPORTANT NOTE: Importing test data via CSV overwrites any values corresponding to matching datasets and variables, including blank values.
- Export your test data
- Click Export CSV to export your datasets to a CSV file. In the Export CSV dialog, select the dataset(s) you would like to export, and then click Export.
User Permissions
Test data is specific to each project in your TestRail instance. Administrators use roles and permissions to manage how testers can use this test data. You can create roles such as Lead, Tester, and Guest and apply permissions based on what actions a person in each role performs.
The Test Data and Variables permission highlighted below affects how your user roles in each project are able to manage this data.
- Add/Edit: Allows users to add and edit variables and datasets within the Test Data management area.
- Delete: Allows users to delete variables and/or datasets entirely.
Using Variables in Test Cases
When you have datasets and variables inside of your project, you can then use this data across your project’s test cases. When editing a test case, you can insert a variable into any markdown-enabled text, string, and separate step field, including shared test steps.
TestRail does not support using variables in any fields not mentioned above, in test case titles, or any in fields outside of test cases (Section names, Test Plan descriptions, etc.).
Insert a Variable into a test case
When editing a test case, you can quickly insert a variable name by typing the name directly into the field, surrounded by percent symbols ‘%’, or by using the variable button to select an existing variable.
Using Variables in Test Runs
When a test run utilizes a dataset and test cases in the test run call variables, TestRail automatically uses the associated dataset values inside of the run’s tests. These values are highlighted in your test, allowing you to quickly identify which variable is in use:
IMPORTANT NOTE: Open test runs always retrieve the current value of the dataset and variable. Editing the values inside of your datasets affects any tests in open test runs. Closing a test run or test plan captures and preserves the dataset values at the time of closure, which creates a static record. You can configure test runs to utilize datasets by clicking Edit on each one and changing the dataset from the drop-down option, as shown in the following image:
You also can configure test runs inside of test plans to utilize datasets by clicking Description, References & Data, and then click the drop-down menu to choose the dataset, as shown in the following image:
Test Parameterization API Endpoints
TestRail’s test data feature is also supported by the TestRail API. The following API endpoints are available to view and manage your test data:
- get_dataset
- get_datasets
- add_dataset
- update_dataset
- delete_dataset
- get_variables
- add_variable
- update_variable
- delete_variable
Please review our API documentation for further details.
Jira Integration Improvements
Inside TestRail, testers on all TestRail plans now have the ability to update linked Jira issues without needing to switch between tools. When hovering over a linked Jira issue ID inside TestRail, testers will now see an Edit button for making updates to the Jira entity. We’ve also added a References field to Test Plans, which will give testers the ability to link an entire Test Plan in TestRail to external requirements, user stories, or other entities. This field will be available for any integrated tool, including Jira.
The TestRail Integration for Jira add-on received a few updates for this release as well. To accompany support for linked Test Plans in TestRail, the add-on now has a Plans tab available for viewing Test Plan data. We also added a Milestones tab to the add-on, to support the available References field in TestRail’s milestones. Finally, the TestRail Integration for Jira add-on will load inside Jira issues in a collapsed view, giving Jira users more screen space if they aren’t interested in linked TestRail data.
Editing Jira Issues
If your project is integrated with Jira using the Jira Defect Plugin, testers will now be able to edit a linked Jira issue. To edit the Jira entity, simply hover over the linked ID and press the Edit button available in the hover window. Pressing the Edit button will open a form, similar to the Push dialog for creating new Jira issues.
Test Plan References
To allow greater flexibility for users, TestRail now has a References field available for Test Plans. This field will be available for any Test Plan, regardless of the integrated tool. With the addition of References on Test Plans, TestRail now supports linking Test Results (using the Defects field), Test Cases, Test Runs, Test Runs inside Test Plans, Test Plans, and Milestones to external entities, such as User Stories, Bugs, or Epics.
New Tabs on the TestRail Integration for Jira Add-on
The TestRail Integration for Jira add-on now has tabs for both Milestones and Test Plans, allowing Jira users to retrieve the most recent data from their TestRail instance and navigate directly to the linked TestRail item.
Other Minor Changes
- The Jira SOAP Defect and Reference plugins will no longer be distributed with TestRail’s installation files.
- The TestRail Integration for the Jira add-on will load in a collapsed view inside Jira issues. This provides more screen space to Jira users who aren’t interested in TestRail information and reduces the load on the connected TestRail instance.
Getting the Updated TestRail Integration for Jira Add-on
Jira Cloud users will automatically receive the updated add-on through Jira.
Jira Server administrators can download the latest version of the add-on through the Jira Administration area, or the Atlassian Marketplace.
Getting TestRail 7.6 on Early Access
You can start a 14-day free trial of TestRail here (cloud or server): https://secure.gurock.com/customers/testrail/trial/. Once your instance is ready, send a request here to get your instance upgraded to the Early Access track.
If you want to create a subscription for TestRail Cloud, you can do so from within TestRail via Administration > Subscription. If you want to order TestRail Server licenses, you can do so from our website here: https://secure.gurock.com/customers/shop/annual/purchase/
Registered customers can download the full version from our customer portal: http://www.gurock.com/portal/
TestRail Cloud
TestRail Cloud instances are automatically updated according to the currently chosen release track. Read more about Quarterly Release release tracks here. You can check your version via the TestRail Help & Feedback > About TestRail menu item.
TestRail Server
If you are using the on-premise version of TestRail, you can update to the new version as usual by installing it over your existing TestRail installation (there’s no need to uninstall your existing installation). The database upgrade wizard is automatically started when you access TestRail with your web browser. Please see the update instructions for details: http://docs.gurock.com/testrail-admin/installation-upgrading/
If you’re using Docker, please refer to the instructions here: http://docs.gurock.com/testrail-admin/installation-docker
Please ensure you make a backup of your current on-premise installation before upgrading to the new version.
TestRail Enterprise
If you’d like to learn more about SSO, Project Level Administration, Test Case Reviews, or any of the other features on TestRail Enterprise, please email us for a trial or quote via [email protected], or you can use the contact form here: https://secure.gurock.com/customers/support/.
Not sure which TestRail plan you’re on? Reach out and we’d be happy to help. Learn more about Quarterly Release release tracks here to receive TestRail 7.6 now.
