How to Integrate Selenium and TestRail

What is Selenium?

Selenium is an open-source tool used to automate web-based applications on various browsers and environments.

What is TestRail?

TestRail is extensive web-based test case management software used to effectively manage, track, construct and organize your software testing efforts.

Integration between Selenium and TestRail defines the result of a unique linked Test ID in TestRail and depends on the result a user is getting from the Selenium automation suite.

The following are the two ways in which you can update your TestRail results after a test case execution:

  • Create a TestNG listener that will listen for test results and call the TestRail API to update it with any results it finds
  • Call the TestRail API inside the test method to update results

Two methods to update test results are:

  • add_result
  • add_result_for_case

The difference between the two is that while “add_result” takes the Test ID, “add_result_for_case” takes the Case ID and Run ID. We can use any method depending on the automation approach chosen.

An advantage of using “add_result_for_case” over “add_result” is that while creating a test run as per the iteration, the test ID changes corresponding to the test run ID, but our test case ID remains the same. This makes it easy to maintain the test case ID and run the automation suite.

Why an integration between TestRail and Selenium is required:

In a real-time project, automation runs one iteration behind schedule. Therefore, it is necessary to add around 10% of manual test cases in each iteration (which will be automated in next iteration).

Let’s take an example of an iteration (sprint32):

Total number of test cases in TestRail = 1000

Total number of test cases added in sprint32 = 100          <== Manual Test Case

Total number of automated test cases in TestRail = 900          <== Automated Test Case

During regression testing of sprint32, the regression automation suite has both manual and automated test cases. By integrating TestRail and Selenium, we can easily update the status of the automated test case in TestRail with detailed failure messages and screenshots. The testing team can then update the remaining 10% manual test cases. This approach saves both time and the manual effort required to update the status in TestRail.

Advantages of TestRail and Selenium Integration:

  • Saves time and manual effort while updating the test case status
  • Provides an estimate of how many manual test cases have been automated and integrated with TestRail so they can be marked as automated for future reference
  • Gives a real picture of the release before starting manual regression testing
  • Helps in streamlining the testing process
  • Helps in verifying RTM (Requirement Traceability Matrix) in JIRA

Sample Test Code :

package test;
import java.io.IOException;
import java.util.HashMap;
import java.util.Map;
import org.json.JSONObject;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.firefox.FirefoxDriver;
import org.testng.Assert;
import org.testng.annotations.Test;
import util.APIClient;
import util.APIException;
public class testgoogle {

    public static String TEST_RUN_ID                = <Test RUN ID>;
    public static String TESTRAIL_USERNAME          = <Test Rail Username>;
    public static String TESTRAIL_PASSWORD          = <Test Rail Password>;
    public static String RAILS_ENGINE_URL           = <Test Rail URL>;
    public static final int TEST_CASE_PASSED_STATUS = 1;
    public static final int TEST_CASE_FAILED_STATUS = 5;
    WebDriver driver;
    
    @Test(description = "Verify that Title appearing on the webpage is as expected. TC_ID=C120254")
    public void verifyTitle() throws Exception, APIException {
        driver= new FirefoxDriver();
        driver.get("https://www.google.co.in");
        String actualTilte = driver.getTitle();
        if (actualTilte.contains("Google")) {
            Assert.assertTrue(actualTilte.contains("Google"));
            testgoogle.addResultForTestCase("120254", TEST_CASE_PASSED_STATUS, "");
        }
else{
		testgoogle.addResultForTestCase("120254", TEST_CASE_FAILED_STATUS, "");
	}

        driver.quit();
    }


    public static void addResultForTestCase(String testCaseId, int status,
            String error) throws IOException, APIException {

        String testRunId = TEST_RUN_ID;

        APIClient client = new APIClient(RAILS_ENGINE_URL);
        client.setUser(TESTRAIL_USERNAME);
        client.setPassword(TESTRAIL_PASSWORD);
        Map data = new HashMap();
        data.put("status_id", status);
        data.put("comment", "Test Executed - Status updated automatically from Selenium test automation.");
        client.sendPost("add_result_for_case/"+testRunId+"/"+testCaseId+"",data );

    }

}

Screenshot of TestRail before execution of automation suite:

testrail-before-execution
Code Snippet:

code-snippet

Screenshot of TestRail after execution of automation suite:

testrail-after-execution

Sample API methods :

There is a Java API binding available at: https://github.com/gurock/testrail-api

Documentation for the API is available here.

You instantiate the API connection in Java with:

import com.gurock.testrail.APIClient;
import com.gurock.testrail.APIException;
import java.util.Map;
import java.util.HashMap;
import org.json.simple.JSONObject;

public class Program
{
    public static void main(String[] args) throws Exception
    {
        APIClient client = new APIClient("http://<server>/testrail/");
        client.setUser("..");
        client.setPassword("..");
    }
}

Here’s an example of a GET request:

APIClient client = new APIClient("http://<server>/testrail/");
client.setUser("..");
client.setPassword("..");
JSONObject c = (JSONObject) client.sendGet("get_case/1");
System.out.println(c.get("title"));

And here’s a POST request:

Map data = new HashMap();
data.put("status_id", new Integer(1));
data.put("comment", "This test worked fine!");
JSONObject r = (JSONObject) client.sendPost("add_result_for_case/1/1", data);
Tarun Gupta

Tarun Gupta

Module Lead
One Response to “How to Integrate Selenium and TestRail”
  1. Kristine on

    Hi Tarun, I am new to Selenium and still a newbie in automation testing. How do I instantiate the API connection in JAVA with my Selenium? I’m using Selenium with Eclipse, I’ve already installed TestNG too. Now, I’m stuck as to how would I integrate TestRail’s API to TestNG/Selenium instance. Looking forward for your reply, thank you so much.

    Reply
Leave a Reply

Related Posts

The Art of Building Rapid and Valuable Proof of Concepts (PO... Clients and stakeholders want results. They want assurances that their investment is well spent and they're building the right product. The industry h...
Getting Started with TestNG (Next Generation) TestNG (Next Generation) is a testing framework that is influenced by both JUnit and NUnit, but has more powerful features and is easier to use. It is...
Automated Testing with Salesforce and Selenium Salesforce is a leading cloud-based CRM that is used by FCA, GE, Coca-Cola, Philips, and other major brands and companies. Being a widely accepted ...
An Introduction to Localized Testing The world we live in has hundreds of languages written and spoken. For many languages, there are multiple locale variants that are popular around the ...
Mobile Automation Testing 101 – Tools, Frameworks and ... In the fast paced software development industry, mobile automation testing has become indispensable. The real value of mobile automation testing is re...