Mobile – iOS Automated Testing with UIAutomation

Background

UI Automation testing is an important feature for mobile application developers  that was introduced in iOS4. It is supported by a new instrument object called “Automation.” It’s quite suitable for UI testing of productivity style applications and is both a probe for Instruments as well as a JavaScript library provided by Apple to exercise and validate a running application. What is Instruments and what is it used for? Here is a quick overview:

• It is a powerful tool you can use to collect data about the performance and behavior of one or more processes on the system
• Lets you gather widely disparate types of data and view them side by side
• Each instrument collects and displays a different type of information, such as file access, memory use, and so forth

Automation instrument works from scripts (written in JavaScript). It simulates/fires required events on target Application. Test script must be a valid executable JavaScript file accessible to the instrument on the host computer. You can launch Instruments by opening Xcode and selecting
Xcode > Open Developer Tool > Instruments.

How to…

… Prepare application

First, you need to do a little groundwork to prepare your application to work with the automation tool. The UI Automation library relies on accessibility information in your UI, so adding a little accessibility info to your UI will make testing it easier1. More specifically, the UI Automation framework looks for the accessibilityLabel property of your UIViews. If you’ve built any web applications, this is somewhat akin to sprinkling id attributes in your HTML markup so that you can find particular DOM nodes easily with JavaScript.

For views constructed in Interface Builder, you can set this property in the Inspector in the “View Identity” tab (the fourth one). Note that not every kind of view provides accessibility configuration in Interface Builder 2. You need to enable Accessibility and you need to provide a Label value. You’ll use these later in your tests to identify particular views.

If you aren’t using IB or your view doesn’t expose accessibility information graphically, you can still set it manually in your code. You can set the is Accessibility Element and accessibility Label properties to get the same effect:

UITableViewCell *)tableView:(UITableView *)tableView cellForRowAtIndexPath:(NSIndexPath *)indexPath {
static NSString *CellIdentifier = @"Cell";
   
UITableViewCell *cell = [tableView dequeueReusableCellWithIdentifier:CellIdentifier];
if (cell == nil) {
cell = [[[UITableViewCell alloc] initWithStyle:UITableViewCellStyleDefault reuseIdentifier:CellIdentifier] autorelease];
  }
   
  // cell configuration elided
   
  cell.isAccessibilityElement = YES;
  cell.accessibilityLabel = @"user name"
  return cell
}

… Writing Your Tests

The next step is to write your first test in JavaScript3 in your editor of choice. This step is a bit like being dropped off in a field somewhere with no map, tools or supplies and being told you need to build a house. There’s no built-in structure for your tests. When you execute a test script, Instruments will run it from beginning to end in linear fashion.

To get started you need to get a reference to the running application from which you can access all the other parts of the app. Put these two lines of boilerplate at the top of your test:

target = UIATarget.localTarget();
application = target.frontMostApp();

The UIATarget is your primary portal into the application running on the device or simulator. It acts as a sort of proxy for the user of the application and is the object you interact with when you want to perform operations on the device such as fiddling with volume controls, shaking, or performing user gestures. The application object (a UIAApplication instance), gives you access to the top-level structure of your application for things like the navigation bar, tab bars, and the main window.

The UIAApplication class has a mainWindow() method that returns a UIAWindow instance. This class is an extension of UIAElement which nearly all of the other UI-related classes derive from. This class provides access to things like the parent view, child views, links, pickers, sliders, table views and nearly anything else you can imagine.

While Apple didn’t provide much high-level information about UI Automation in the form of a Programming Guide, the UI Automation Reference Guide is worth bookmarking or keeping handy in PDF format. This is an essential reference that describes the JavaScript API in great detail. Check the Xcode documentation.

So let’s look at a real example 4. Let’s say we have an application connects to a variety of popular web sites. When the user launches the app for the first time, we want to detect that they don’t have any accounts configured and prompt them to create one. Our screen might look something like this:

We want to verify that when we launch the application we see this screen. Once we’ve verified this, we want to create a twitter account, so we’ll select the “twitter” table cell. Here’s the test script so far:

test("Initial screen", function(target, app) {
  // check navigation bar
  navBar = mainWindow.navigationBar();
  assertEquals("Add Account", navBar.name());
  button = navBar.leftButton();
  assertEquals("Back", button.name());
 
  // check tables
  table = mainWindow.tableViews()[0];
  tableGroup = table.groups()[0];
  assertEquals("What type of account?", tableGroup.name());
 
  assertEquals(4, table.cells().length);
  ["facebook", "flickr", "github", "twitter"].each(function(i,e) {
    assertNotNull(table.cells().firstWithName(e));
  });
   
  // more to come...
});

The test, assertNotNull and assertEquals functions are ones I wrote to add some structure and high-level validation to the testing process. I found the way the UI Automation suite does test case declaration to be really verbose and crude so I boiled test declaration into something simpler. First, the test() method takes a string title and a function to execute as the body:

function test(title, f, options) {
  if (options == null) {
    options = {
      logTree: true
    };
  }
  target = UIATarget.localTarget();
  application = target.frontMostApp();
  UIALogger.logStart(title);
  try {
    f(target, application);
    UIALogger.logPass(title);
  }
  catch (e) {
    UIALogger.logError(e);
    if (options.logTree) target.logElementTree();
    UIALogger.logFail(title);
  }
};

This handles the boilerplate of getting the target and application references. It also provides some structure around UIALogger.logStart(), UIALogger.logPass() and UIALogger.logFail(). These three methods are how the UI Automation framework demarcates tests. However I found that using if checks and calling logFail() really muddied the tests, so I wrote some assertion methods that throw exceptions and the test structure automatically catches them and logs the test as a failure.

Here are the assertion methods:

function assertEquals(expected, received, message) {
  if (received != expected) {
    if (! message) message = "Expected " + expected + " but received " + received;
    throw message;
  }
}
 
function assertTrue(expression, message) {
  if (! expression) {
    if (! message) message = "Assertion failed";
    throw message;
  }
}
 
function assertFalse(expression, message) {
  assertTrue(! expression, message);
}
 
function assertNotNull(thingie, message) {
  if (thingie == null || thingie.toString() == "[object UIAElementNil]") {
    if (message == null) message = "Expected not null object";
    throw message;
  }
}

Next we want to fill out the text fields in the next view with our twitter credentials:

We extend the test code above with the next set of stimuli and assertions:

// create a new account
  table.cells().firstWithName("twitter").tap();
  mainWindow = app.mainWindow();
  table = mainWindow.tableViews()[0];
 
  userName = table.cells().firstWithName("user name");
  assertNotNull(userName, "No username field found");
  userName.textFields()[0].setValue("mrfoobar");
 
  password = table.cells().firstWithName("password");
  assertNotNull(password, "No password field found");
  assertNotNull(password.secureTextFields()[0], "No text field found for password");
  password.secureTextFields()[0].setValue("sekret");
   
  finish = table.cells().firstWithName("finish");
  assertNotNull(finish, "No finish button found");
  finish.tap();
   
  // more to come...

The first line above finds the “twitter” cell by calling the firstWithName() method on the collection of cells in the table view. This table view was generated programmatically, so that “twitter” label came from setting the cell’s isAccessibilityElement property to YES and its accessibilityLabel property to @”twitter”. Next we touch that table cell via the tap() method to navigate forward in our application.

The next two stanzas validate that we have user name and password fields, and also fills them out using the setValue() method. Finally we look for the cell that contains the “Finish” button and tap it to finish creating our account.

The last bit of testing to do here is to validate that we did indeed land on our settings screen. So we add a few more assertions to our test:

// validate settings screen
  finish.waitForInvalid();
  mainWindow = app.mainWindow();
  assertEquals("Settings", mainWindow.navigationBar().name());

Here we call the waitForInvalid() method on the finish button object. Without this, I found that the other code executed too quickly before the window transition completed. The waitForInvalid() will wait (up to a configurable timeout value) for that object to be invalidated. Our last assertion is that the title in the navigation bar is “Settings”.

… Running Tests

Now that we have a test, we need to run it in Instruments. When you launch Instruments, choose “Automation” from the iPhone templates.

In the details for the Automation instrument, choose your script via the “Choose Script…” drop-down. Next, you need to choose your target from the toolbar. Once you have these setup, you run the flow by (somewhat counter-intuitively) hitting the record button (or using ?+R). This will launch your application, wait a few seconds and then run your test. Note that even if your test completes, Instruments will keep running your application. To formally end the test toggle the red record button, or hit ?+R again.

Tests are listed in the detail view, with the test name in the “Log Messages” column. If your test passes, the “Log Type” value will be “Pass” in comforting green. If your test fails, the value is “Fail” in scary red. You can expand the test to see the details of what happened. Because of the way I wrote the test structure above, any test failures automatically log the view hierarchy for inspection.

… Debugging tests

When things go wrong with your tests you don’t have a lot to look at. However there are a few things you can inspect to try to figure out what’s broken. First, log the element tree (via UIATarget.logElementTree()) liberally and often. Although view hierarchy is a tree and it’s presented in a tree control, it’s “flattened-out” out for display purposes. However, the numeric prefix listed in each row indicates the level that node is at, so you can infer parent-child relationships:

In this case, the tree looks something like this:

UIATarget
      UIAApplication
         UIAWindow
            UIATableView
               UIATableGroup
                  UIAStaticText

You don’t always have to traverse the entire tree, node-by-node, to find what you’re looking for. Take a look at the various methods on the UIAElement class and experiment a bit.

You can also log any message you choose using any of these log methods on the UIALogger class:

  logDebug()
    logMessage()
    logWarning()
    logError()

All of these just log messages of varying severity, but don't affect whether or not the test is marked as passing or a failure.

Building UI Automation

While Apple may not have provided much support for automation testing beyond a JavaScript API and a reference doc, they (wisely) left the whole mechanism open to easy extension. You can import other JavaScript into your tests with the import statement. I’m not 100% sure how smart that statement is with regard to paths, but I found that if you put supporting scripts in the same directory as the test script you’re executing, Instruments will automatically pick it up.

So I took the extensions I mentioned earlier, plus a few more, and put them into a small, but growing, JavaScript library I call tuneup_js.

You can simply copy all of the JavaScript files into the same directory as your automation tests. For my project, I created a separate directory called “Automation Tests” and dumped everything in there. This is pretty crude and I’d like to improve it, but I need to spend some more time figuring out the nuances of how Instruments handles paths in import statements.
UI Automation characteristics:

  • Automates UIKit based applications
  • Touch based
  • iPhone, iPod touch and iPhone Simulator
  • Integrated in Instruments
  • Accessibility based
  • JavaScript automation scripts

Tips to simplify your life

Tune-up – write code using:

 var target = UIATarget.localTarget();

    var app = target.frontMostApp();

    var window = app.mainWindow();

Import external scripts – you can also see how to reference one script from another, with #import directive.

 #import "Test1.js"
    #import "Test2.js"

Power of the command line”  (RECOMMENDED) – If you want to automate your scripts, you can launch them from the command line
– you will need your UDID and type on a terminal

instruments -w your_ios_udid -t /Applications/Xcode.app/Contents/Developer/Platforms/iPhoneOS.platform/Developer/Library/Instruments/PlugIns/AutomationInstrument.bundle/Contents/Resources/Automation.tracetemplate name_of_your_app -e UIASCRIPT absolute_path_to_the_test_file

When things don’t work, add UIATarget.delay(1);”

How to use it:

Step 1

  • Make sure that you have the latest iOS SDK (4.0+) installed | iOS SDK
  • Familiarize yourself with Instruments | Instruments User Guide

Step 2
Ensure accessibility is enabled and all the UI controls and views in your application are tagged with unique accessibility labels which makes it easy to access them in our test scripts. Using Interface Builder, you can set an element’s accessibility status and provide custom content for the label.

Step 3
In essence, your test script is an ordered set of commands, each of which accesses a user interface element in your application to perform a user action on it or to use the information associated within it. All the user interface elements in your application are represented to the script through an ordered hierarchy of objects defined by the UIAElements class and its subclasses.
To reach a specified UI element, the script simply calls down the element hierarchy, starting with the top level target object obtained by calling UIATarget.localTarget()

Step 4

  • Once you are ready with the test script. Compile your app in the debug mode and create a simulator build.
  • Launch Instruments and select Automation from iOS Simulator templates.
  • Select your target application from “Choose Target” option on the top.
  • Select the test script from “Choose Script” option on the left window.
  • Hit the red record button once done. It will fire up the simulator with your target app and start executing the tests.

Update:
– If you are facing any issues running UIAutomation on the iPad, refer to UIAutomation on iPad.

Documentation:

Presentation made by Michael Creasy QA Manager, UI Automation & iPhone SDK UI Automation Reference Collection

https:// developer.apple.com/wwdc/iphone/library/documentation/DeveloperTools/Reference/UIAutomationRef/index.html

Instruments User Guide

https:// developer.apple.com/wwdc/iphone/library/documentation/DeveloperTools/Conceptual/InstrumentsUserGuide/index.html

Apple Developer Forums

http://devforums.apple.com

Also:

http://sgleadow.github.com/blog/2011/10/26/which-automated-ios-testing-t…

http://developer.apple.com/library/ios/#documentation/DeveloperTools/Ref…

 

Rares Irimies

Rares Irimies

QA Engineer

Rares Irimies is a QA Engineer in 3Pillar’s Cluj, Romania office who is currently working on a Network Security application. Rares is responsible for conducting functional testing of front-end and back-end features that analyze network anomalies, performing traffic interpretation using Wireshark, test case creation, and manual and automation testing using a Perl-based automation framework.

3 Responses to “Mobile – iOS Automated Testing with UIAutomation”
  1. Debjit Mukherjee on

    I want to take data from number spreadsheet for testing.How to do that?

    Reply
  2. arif on

    Awsome stuff Rares Irimies.Its very usefull for ios automation guys .

    Thanks you very much

    Reply
  3. Habin Prasad on

    Hi, I would like to create an automation framework using UIAutomation. Can you help me in it

    Reply
Leave a Reply

Related Posts

Designing the Future & the Future of Work – The I... Martin Wezowski, Chief Designer and Futurist at SAP, shares his thoughts on designing the future and the future of work on this episode of The Innovat...
The 4 Characteristics of a Healthy Digital Product Team Several weeks ago, I found myself engaged in two separate, yet eerily similar, conversations with CEOs struggling to gain the confidence they needed t...
Recapping Fortune Brainstorm Tech – The Innovation Eng... On this episode of The Innovation Engine, David DeWolf and Jonathan Rivers join us to share an overview of all the news that was fit to print at this ...
4 Reasons Everyone is Wrong About Blockchain: Your Guide to ... You know a technology has officially jumped the shark when iced tea companies decide they want in on the action. In case you missed that one, Long Isl...
The Connection Between Innovation & Story On this episode of The Innovation Engine, we'll be looking at the connection between story and innovation. Among the topics we'll cover are why story ...