This article is part of a series of blog posts for Streamlining test automation with TestRail, which aim to guide you through centralizing your test automation efforts in TestRail, as per the diagram below.
Most teams who integrate their automated tests with TestRail trigger and execute their automated tests outside of TestRail (e.g. as part of a continuous integration system) and use TestRail’s API to submit the test results.
However, if you want to centralize the regular testing activities on TestRail and minimize navigation between systems, you can also implement a mechanism to trigger your Jenkins test runs directly from TestRail by using UI scripts. In this final article of this blog series, we’ll guide you through that process by using one of the UI Scripts examples we provide on the TestRail Customizations GitHub repository. This UI Script integrates a Jenkins builds widget within the Test Runs & Results page on TestRail. By implementing this last step, you can execute automated tests and retrieve their results without ever leaving TestRail, finally closing the circle.
Configuring Jenkins
To allow the TestRail UI to make direct requests to your Jenkins instance, you need to allow Cross-origin resource sharing (CORS) on the Jenkins end. Although this is usually considered a security risk, with the right precautions, you can leverage this mechanism to allow TestRail to access your Jenkins jobs data without increased risk. To do so, you should allow CORS to your trusted TestRail domain by following the steps below.
1. Go to Manage Jenkins > Plugin Manager and search for and install the CORS support for Jenkins plugin
2. Go to Manage Jenkins > Configure System
3. At the bottom of the page, on the CORS Filter section, insert the required parameters
4. Click Save to accept the changes
Configuring the UI Script
You can download the Jenkins Builds UI Script from the TestRail Customizations GitHub. Then, to add the UI script, go to TestRail’s administration area under Administration > Customizations > Add UI Script and simply copy and paste the whole UI script to the Configuration text box. In the examples below, we have UI script samples you can use but if you want to learn how to develop your own custom scripts or further customize these samples, please check the UI scripts documentation page..
There are only a couple of values you have to change on the script to properly configure it to reach your Jenkins instance and retrieve data from the desired jobs. The first thing is setting the host variable to your Jenkins URL. Then you need to change the examples on the jobs variable. You can have any number of jobs. The name of each job can be anything you want shown on the trigger button and the path is the relative path to the job, which you can copy from your browser when you navigate to the job on Jenkins.
/* CONFIG */
const host = "https://INSTANCE.testrail.io";
const username = null; // Enter username and key only if you fully trust your organization members as this may pose a security risk
const key = null; // Generate an API key on your Jenkins user account
const requestInterval = 10000; // Time between requests to Jenkins
const project = "PROJECT NAME"; // Name of the project to load the triggers for (use value null to enable for all projects)
const jobs = [
{
"name": "Run Automated Tests",
"path": "job/Automated%20Tests%20Project"
}
];
Note: You can enter the Jenkins username and key if you fully trust all the TestRail users in your organization, but be aware that this poses a security risk since the UI script (and credentials) will be available in plain text on the browser. We recommend leaving these variables null and letting users configure their credentials through the UI.
The UI Script in action
As mentioned, the UI script adds a Jenkins integration section to the right-hand panel, which can be configured to display your automated tests jobs (or any other jobs if you wish). On the panel, we can see a new section called Automated Test Triggers and the jobs that you configured on your UI script. For each job, there’s a trigger button that will start a new build and also links to the 3 latest builds, using a color code that indicates the status of the build. In this case, red for failure, green for success, and blue means it’s still running.
Click on Run Automated Tests and you will trigger your job on Jenkins through its API. The new job build will be displayed on the list below the trigger button in blue while it is running and then either red or green, depending on the final build result. The builds list is made of links so you can easily navigate to Jenkins if you want to dig into more build details.
Final result
By implementing the last step in this series, you have just closed the circle and achieved the workflow in the diagram below.
With this workflow in place, triggering and accessing automated tests information becomes part of TestRail. Any user who is performing his daily activities in TestRail can start an automated tests suite, and receive the test results as a test run as soon as it is finished on the CI tool, without ever needing to navigate outside TestRail.
Now that you have centralized your test results in TestRail, you can check the results of your automated test runs, along with the error messages for failed tests. You can also aggregate both your manual and automated testing information on reports. By doing so, you can get solid insights on your app’s test coverage and even track test automation progress.
