How To Manage QA Activities with Rally

The purpose of this blog is to describe how we can manage activities like test set, test cases from creation to execution, and other QA activities using the tool Rally. While doing testing in my current project, I was facing challenges when pulling out a Test case execution report for each release in the iteration.

I have used the test set feature, which helps me to manage test cases efficiently for regression testing and to get the report of each release.I want to share the different ways of Test case creation using Rally and how we can use Rally Add-in to import test cases from MS-Excel directly.

I also want to share how Rally can help us keep the quality track and the history of the build or release, as well as how Rally can be used automatically (having the tests updated with results from running automation tests).

What is Rally?

Rally is an agile project management solution that provides release planning, iteration planning, the ability to schedule all or parts of requirements from release backlog, and the ability to manage other QA activities.

What is a Test Set?

A test set is a container for test cases, which can be run in an iteration or release. Test sets allow you to group and schedule regression tests in an iteration or release. For example, a test set could include smoke, configuration, system, and so on. Once grouped, you can run test cases together.

Steps to Manage Test Cases and other QA Activities in my Current Project

Create work product or story

The first step in the creation of test cases is to create the work product or story. Below are the steps to create this:

Prerequisite: the user should have a valid Rally credential

Click on Plan -> User Stories then Action -> New User Story

rally_tool_1

Create Test Plan

To create a test plan, follow the steps below:

From Quality, select Test Plan. From Actions, select New Test Folder. On the Create Test Folder editor, enter the name of your new test folder.

Finally, click Save & Close.

rally_tool_2

Create Test Cases

There are three ways to create test cases:

1. Create test cases as a stand-alone: The first way to create the test cases is as stand-alone. Use the following to do so: Click the Quality tab -> Test Cases, from Actions -> Click New Test Case. The following screen will open to enter test case detail with the option to link the test case with a work product.

rally_tool_3

2. Create test cases by importing directly in Rally using a .csv file. The second way to create test cases is by importing them from a .csv file. Below are steps to import the test cases from .csv file:

  1. Create an Excel file that’s correctly formatted and convert this excel file into a .csv file
  2. Click the Quality tab -> Test Cases, From Actions -> Click on Import Test Case
  3. Click browse button and select .csv file and click on Import button. Imported test cases can be verified under Quality tab -> Test Plan

3. Create test cases using the Rally add-in for Excel. The third way to create test cases is by importing a test case directly from Excel using “Rally Add-in for Excel.” Below are the steps to import the test cases using Add-in:

  1. Install the Add-in: download RallyAddinForExcel_2.3.3.msi. To install, run the .msi file
  2. Set up the Add-in within Excel: Once you install the Add-in, a new Rally tab will display in the Excel ribbon
    rally_tool_4
  3. Click on the Connect to Rally icon in the Excel file and enter the Rally server URL (defaul is rally1.rallydev.com), username, and password
    rally_tool_5
  4. After entering a valid username and password, click on the Connect button. After a few seconds, the Import to Rally button will enable an Excel file to import the test cases
  5. Click on the Import to Rally button and the screen below will open to import the test cases
  6. Select type of Test Case
  7. Click on the Validate button to make sure that the import file format is valid. As a user, click on the Validate button, and if everything is fine, the message “Import data is valid” will display
    rally_tool_6
  8. Once the test case format is validated, the next step is to import the test cases in Rally. Click on the Import button shown in the screenshot above
    rally_tool_7
  9. Once the import is completed, the user can verify the imported test cases under Quality -> Test Plan -> Test Folder
    rally_tool_8
  10. The user can undo the imported test cases by clicking on the Undo button
    rally_tool_9

Create Release Cycle

The next step is to create the release cycle. To create the release cycle, click on the Plan -> Time boxes. Then click on “Add with detail;” the following screen will open for the user to enter release data.

rally_tool_10

In the screen below, we have created two cycles/releases for the iteration. We will execute test cases for two cycles (this varies from project to project) for each story.

rally_tool_11

Create Test Sets

The next step is to create the test set. A test set is a container for test cases that can be run in an iteration or release. To create test sets, click the Track tab, then select Iteration Status or Release Status.

  1. On the Iteration Status page, select Test Set from the drop-down menu and click the Add New button
    rally_tool_12
  2. On the Release Status summary page, click Actions and then select Schedule New Test Set
    rally_tool_13

Linking Between Test Set, Release, and Iteration

The screen below will open when a user clicks on Schedule New Test Set. Here a user can link between test sets, releases, and iterations.

rally_tool_14

Copy Test Cases from Test Plan Against Created Test Set

The next step is to copy the test cases in the test set by clicking on the icon against the test set (see the screenshot below).

rally_tool_15

The screen below will open to choose the test cases from the test plan. Once the test cases are selected, they will display under the particular test set.

rally_tool_16

Execution of the Test Set and Defect Reporting

The last step is test set execution, where a user can run the test set for two releases or cycles.

  1. To run test sets, click the Track tab, then select Iteration Status
  2. Click the row action icon bin gear menu next to the test set and select Run Test Set
  3. The test run page displays all test cases and their related test steps contained within the test set. Use this page to run each test case within the test set.

Below are the results after the test set execution for Release #1. Here we can see that 14 test cases passed and 5 test cases failed; when a user expends the TS10 in the screenshot below, all passed and failed test cases will be displayed.

rally_tool_17

In the screenshot below, we can see defect (DE5035, DE5036) is associated with a failed test case.

rally_tool_18

Below are the results after the test set execution for Release #2. Here we can see that all 19 test cases passed.

rally_tool_19

Conclusion

With this blog, we have understood how we can use test sets to execute test cases using the tool Rally. There are many other tools for test case management, like Jira, TEst TRack, and Test Rail, but Rally is one of the best tools for managing both test cases and defects in a single tool.

We have also understood how we can pull out test case execution reports for each release, and how we can use Rally Add-in to import test cases from MS-Excel directly.

 

Kailash Pathak

Kailash Pathak

Module Lead Quality Assurance

Kailash Chander Pathak is a Module Lead Quality Assurance at 3Pillar Global. Kailash has much experience in Telecommunications & Satellite Industries, as well as in Finance domains. Kailash has worked in domains like Healthcare, Education, ERP, and Standards Compliance. He also has expertise in various types of testing such as Functional, System, Regression, and Performance. He is a ISTQB, PRINCE2®, and PMI-ACP® Certified professional. Prior to joining 3Pillar Global, he has worked for Xavient Information Systems.

2 Responses to “How To Manage QA Activities with Rally”
  1. Jacob Sanchez on

    Hi, I have created some test cases using the CSV document provided by Rally, but I would like to know if there is a way to update existing test cases using a CSV format or any other kind of documents instead of manually changing the test cases on Rally.

    Thanks in advance

    Reply
  2. des on

    1. linking the many test cases to user story was suddenly not working.
    2. The adding existing test case in every user story in portfolio item is also not displayed!

    Please help how can i link my existing test cases to user story with clear steps. 🙂

    Thank You!

    Reply
Leave a Reply

Related Posts

Interviews from Industry Summit 2017: the Product Conference On this special double episode of The Innovation Engine, the 3Pillar team interviews many of the speakers who took the stage at Industry Summit 2017. ...
Designing the Future & the Future of Work – The I... Martin Wezowski, Chief Designer and Futurist at SAP, shares his thoughts on designing the future and the future of work on this episode of The Innovat...
The 4 Characteristics of a Healthy Digital Product Team Several weeks ago, I found myself engaged in two separate, yet eerily similar, conversations with CEOs struggling to gain the confidence they needed t...
Recapping Fortune Brainstorm Tech – The Innovation Eng... On this episode of The Innovation Engine, David DeWolf and Jonathan Rivers join us to share an overview of all the news that was fit to print at this ...
4 Reasons Everyone is Wrong About Blockchain: Your Guide to ... You know a technology has officially jumped the shark when iced tea companies decide they want in on the action. In case you missed that one, Long Isl...