How to Integrate Selenium and TestRail

What is Selenium?

Selenium is an open-source tool used to automate web-based applications on various browsers and environments.

What is TestRail?

TestRail is extensive web-based test case management software used to effectively manage, track, construct and organize your software testing efforts.

Integration between Selenium and TestRail defines the result of a unique linked Test ID in TestRail and depends on the result a user is getting from the Selenium automation suite.

The following are the two ways in which you can update your TestRail results after a test case execution:

  • Create a TestNG listener that will listen for test results and call the TestRail API to update it with any results it finds
  • Call the TestRail API inside the test method to update results

Two methods to update test results are:

  • add_result
  • add_result_for_case

The difference between the two is that while “add_result” takes the Test ID, “add_result_for_case” takes the Case ID and Run ID. We can use any method depending on the automation approach chosen.

An advantage of using “add_result_for_case” over “add_result” is that while creating a test run as per the iteration, the test ID changes corresponding to the test run ID, but our test case ID remains the same. This makes it easy to maintain the test case ID and run the automation suite.

Why an integration between TestRail and Selenium is required:

In a real-time project, automation runs one iteration behind schedule. Therefore, it is necessary to add around 10% of manual test cases in each iteration (which will be automated in next iteration).

Let’s take an example of an iteration (sprint32):

Total number of test cases in TestRail = 1000

Total number of test cases added in sprint32 = 100          <== Manual Test Case

Total number of automated test cases in TestRail = 900          <== Automated Test Case

During regression testing of sprint32, the regression automation suite has both manual and automated test cases. By integrating TestRail and Selenium, we can easily update the status of the automated test case in TestRail with detailed failure messages and screenshots. The testing team can then update the remaining 10% manual test cases. This approach saves both time and the manual effort required to update the status in TestRail.

Advantages of TestRail and Selenium Integration:

  • Saves time and manual effort while updating the test case status
  • Provides an estimate of how many manual test cases have been automated and integrated with TestRail so they can be marked as automated for future reference
  • Gives a real picture of the release before starting manual regression testing
  • Helps in streamlining the testing process
  • Helps in verifying RTM (Requirement Traceability Matrix) in JIRA

Sample Test Code :

package test;
import java.io.IOException;
import java.util.HashMap;
import java.util.Map;
import org.json.JSONObject;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.firefox.FirefoxDriver;
import org.testng.Assert;
import org.testng.annotations.Test;
import util.APIClient;
import util.APIException;
public class testgoogle {

    public static String TEST_RUN_ID                = <Test RUN ID>;
    public static String TESTRAIL_USERNAME          = <Test Rail Username>;
    public static String TESTRAIL_PASSWORD          = <Test Rail Password>;
    public static String RAILS_ENGINE_URL           = <Test Rail URL>;
    public static final int TEST_CASE_PASSED_STATUS = 1;
    public static final int TEST_CASE_FAILED_STATUS = 5;
    WebDriver driver;
    
    @Test(description = "Verify that Title appearing on the webpage is as expected. TC_ID=C120254")
    public void verifyTitle() throws Exception, APIException {
        driver= new FirefoxDriver();
        driver.get("https://www.google.co.in");
        String actualTilte = driver.getTitle();
        if (actualTilte.contains("Google")) {
            Assert.assertTrue(actualTilte.contains("Google"));
            testgoogle.addResultForTestCase("120254", TEST_CASE_PASSED_STATUS, "");
        }
else{
		testgoogle.addResultForTestCase("120254", TEST_CASE_FAILED_STATUS, "");
	}

        driver.quit();
    }


    public static void addResultForTestCase(String testCaseId, int status,
            String error) throws IOException, APIException {

        String testRunId = TEST_RUN_ID;

        APIClient client = new APIClient(RAILS_ENGINE_URL);
        client.setUser(TESTRAIL_USERNAME);
        client.setPassword(TESTRAIL_PASSWORD);
        Map data = new HashMap();
        data.put("status_id", status);
        data.put("comment", "Test Executed - Status updated automatically from Selenium test automation.");
        client.sendPost("add_result_for_case/"+testRunId+"/"+testCaseId+"",data );

    }

}

Screenshot of TestRail before execution of automation suite:

testrail-before-execution
Code Snippet:

code-snippet

Screenshot of TestRail after execution of automation suite:

testrail-after-execution

Sample API methods :

There is a Java API binding available at: https://github.com/gurock/testrail-api

Documentation for the API is available here.

You instantiate the API connection in Java with:

import com.gurock.testrail.APIClient;
import com.gurock.testrail.APIException;
import java.util.Map;
import java.util.HashMap;
import org.json.simple.JSONObject;

public class Program
{
    public static void main(String[] args) throws Exception
    {
        APIClient client = new APIClient("http://<server>/testrail/");
        client.setUser("..");
        client.setPassword("..");
    }
}

Here’s an example of a GET request:

APIClient client = new APIClient("http://<server>/testrail/");
client.setUser("..");
client.setPassword("..");
JSONObject c = (JSONObject) client.sendGet("get_case/1");
System.out.println(c.get("title"));

And here’s a POST request:

Map data = new HashMap();
data.put("status_id", new Integer(1));
data.put("comment", "This test worked fine!");
JSONObject r = (JSONObject) client.sendPost("add_result_for_case/1/1", data);
Tarun Gupta

Tarun Gupta

Module Lead
8 Responses to “How to Integrate Selenium and TestRail”
  1. Kristine on

    Hi Tarun, I am new to Selenium and still a newbie in automation testing. How do I instantiate the API connection in JAVA with my Selenium? I’m using Selenium with Eclipse, I’ve already installed TestNG too. Now, I’m stuck as to how would I integrate TestRail’s API to TestNG/Selenium instance. Looking forward for your reply, thank you so much.

    Reply
  2. James Mccartney on

    Hello Tarun, thank you for sharing your tutorial. I am also having issues trying to integrate TestRail with my Selenium test cases. I am able to connect to TestRail with my Selenium script in Java but I am having issues running the test cases. I am using Java and Eclipse.

    Reply
  3. Kent on

    Very nice rundown, Tarun. Getting back up to speed with TestRail integration for an interview and this was very helpful!

    Reply
  4. Priyanka on

    Hi Tarun,

    what is TEST_RUN_ID in code?

    Reply
  5. ragini on

    I am new to Selenium and still a newbie in automation testing. How do I instantiate the API connection in JAVA with my Selenium? I’m using Selenium with Eclipse, I’ve already installed TestNG too. Now, I’m stuck as to how would I integrate TestRail’s API to TestNG/Selenium instance. Looking forward for your reply, thank you so much.

    Reply
  6. Zuan Education on

    The article HOW TO INTEGRATE SELENIUM AND TESTRAIL seems to be more informative. This is more helpful for our Selenium training in Chennai. Thanks for sharing.

    Reply
  7. Amit Sehgal on

    What is the code for RETEST a test case in TestRail using Automation Script like PASSED we have TEST_CASE_PASSED_STATUS = 1 ?

    Reply
  8. abuthaiba on

    Very useful topic, thanks for the article.

    Reply
Leave a Reply