Mobile – iOS Automated Testing with Frank

Frank is an automated acceptance testing framework for iOS applications that’s based on Cucumber. This blog post is meant to provide iOS developers with a mini-tutorial on setting up Frank as an automated acceptance testing framework for iOS applications . Below, I’ll summarize all the information necessary to set up and write tests in Frank.

Introduction to Frank/Cucumber

Frank embeds a http server into the test version of app which will be run when the app gets started. Via this internal http server the cucumber part of frank will send commands to the app to remote control it (taps, button taps, filling texts and so on) or to look for UI elements we want to see on specific views.

The first step is create a “Frankified” version of the app for testing by modifiying a copy of the iOS Apps target configuration in XCode:

  • main gets replaced by a version that start the http server before it creates the usual UIApplicationMain
  • it links to the Frank library
  • and it includes a Frank resource bundle used by the http server

That is described in more detail in the Frank documentation that’s linked in the references section below, so I will not repeat it here. After having a “Frankified” app, we can start writing Cucumber scenarios.

Frank already comes with a number of usable Cucumber steps, making it easier to get started and are also useful as a template for private steps.

Here is a list of the built-in steps:

Then /^I wait to see "([\^\"|]\*)"$/
Then /^I wait to not see "([\^\"|]\*)"$/
Then /^I wait to see a navigation bar titled "([\^\"|]\*)"$/
Then /^I wait to not see a navigation bar titled "([\^\"|]\*)"$/
Then /^I should see a "([\^\"|]\*)" button$/
Then /^I should see "([\^\"|]\*)"$/
Then /^I should not see "([\^\"|]\*)"$/
Then /^I should see the following:/
Then /^I should see a navigation bar titled "([\^\"|]\*)"$/
When /^I type "([\^\"|]*)" into the "(**[\^\"|]*)" text field$/
When /^I fill in "([\^\"|]*)" with "(**[\^\"|]*)"$/
When /^I fill in text fields as follows:$/
And /^I rotate to the "([\^\"|]\*)"$/
When /^I touch "([\^\"|]\*)"$/
Then /I touch the following:/
When /^I touch the button marked "([\^\"|]\*)"$/
When /^I touch the "([\^\"|]\*)" action sheet button$/
When /^I touch the (\d*)(?:st\|nd\|rd\|th)? action sheet button$/
When /^I flip switch "([\^\"|]\*)"$/
Then /^switch "([\^\"|]\*)" should be on$/
Then /^switch "([\^\"|]\*)" should be off$/
Then /^a pop-over menu is displayed with the following:$/
Then /^I navigate back$/

Having quite a number of example steps it is not very difficult to create additional steps. All the built-in steps are just a few lines and easy to understand.

By default Frank uses the “Accessibility Label” to locate UI elements in the app using UISpec. The “Accessibility Label”s original use is to give a hint what a control is for to people with visual impairments. Frank uses it to mark UI elements with a “label” that we can use in our cucumber steps to refer to that UI element.

For example the cucumber scenario using a custom step looks like this:

Scenario:
ask user to login at first startup
Given the app runs for the first time
Then I should be on the "Login" screen
And I should see text fields for:
| Login |
| Password |

The “And” step is using a custom step and it (i.e. UISpec triggered by a call to the embedded http server) will look for two text fields that are marked with the accessibility labels “Login” and “Password”.

Frankify app in XCode 4

Step 1: Install the frank-cucumber gem

Simply run:

sudo gem install frank-cucumber

from the command line to download and install the gem.

Step 2: Add the Frank skeleton directory to your app source code

In the terminal cd to your app’s source code directory (e.g. where the XCode project file lives), and execute:

frank-skeleton

This will add a Frank subdirectory containing Frank’s server code plus some initial cucumber plumbing, after checking with you first.

Step 3: Create a Frankified target

You need to create a separate XCode app target for a ‘Frankified’ version of your app. This Frankified target will link the Frank server component into your app, so that it can be automated.

In XCode, switch to the Project Navigator by hitting Command-1, and then select your project by clicking on it. You should now see your project settings, with one or more targets listed. Right-click on your main app target, and select “Duplicate”. You may be asked if you want to transition to an iPad target. If so, select “Duplicate Only”. You should now see a new target created called ” copy”, or similar. Rename the target to ” Frankified”.

Step 4: Add the Frank server to your Frankified target

Right-click on your project In the Project Navigator at the far left, choose “Add Files To “. Select the Frank directory which you just added to your source directory in the previous step. In the “Add to targets” section at the bottom of the dialog make sure you check only the Frankified target you just created, and uncheck any other targets. Now you can click Add.

Step 5: Add the CFNetwork dependency to your Frankified target

The Frank server uses the CFNetwork framework. If your app isn’t already using it you’ll need to add it as a dependency. In the Project Navigator, select your project again, and select the “Build Phases” tab. Expand the “Link Binary With Libraries” section, and click the + icon. Select “CFNetwork.framework” from the list of frameworks, then click Add.

Step 6: Add necessary linker flags

  • Add “-ObjC” to the “Other Linker Flags” setting
    • Find “Other Linker Flags” in the list of settings. Protip: enter “other” into the “Search in build settings” field to shorten the list
    • Select the Other Linker Flags setting. Click once on the Value column and enter
    • -ObjC
    • Note: If you double click on the Value you’ll get a dialog to help you enter values. You can use it too, just hit “+” and then enter -ObjC there and then press “Ok”.
  • Add “-DFRANK” to the “Other C Flags” setting
    • Find “Other C Flags” in the list of settings. If you used the protip it should be 12th down in the list from the “Other Linker Flags”
    • Edit Other C Flags setting and add -DFRANK
    • Close down the target info window view

Step 7: Add the code to launch Frank into main

We’ll use conditional compilation rather than duplicating the main.m.

Open your main.m file in your project using “File->Open Quickly…”, typing main.m into the field and then pressing enter. Your project’s main.m will be opened in the editor section in Xcode.

At the top of your main.m add in:

#ifdef FRANK
    #include "FrankServer.h"
    static FrankServer *sFrankServer;
    #endif

Before the line “int retVal = UIApplicationMain(argc, argv, nil, nil);” add the following:

#ifdef FRANK
    sFrankServer = [[FrankServer alloc] initWithDefaultBundle];
    [sFrankServer startServer];
    #endif

These lines of code will compile Frank support into the app if “FRANK” is defined, which we did only for the Frankified target as the last thing in the previous step.

Your main.m should look like so:

#import <uikit/UIKit.h>
#ifdef FRANK
#include "FrankServer.h"
static FrankServer *sFrankServer;
#endif
int main(int argc, char *argv[]) {
    NSAutoreleasePool * pool = [[NSAutoreleasePool alloc] init];
#ifdef FRANK
    sFrankServer = [[FrankServer alloc] initWithDefaultBundle];
    [sFrankServer startServer];
#endif
    int retVal = UIApplicationMain(argc, argv, nil, nil);
    [pool release];
    return retVal;
}

Step 8: Test ‘er out!

You should be good to go at this point. Select the Frankified target from the Scheme Selector (at the top, just to the right of the Run and Stop buttons), then hit Run. You should see your app build and launch as normal. But, if you now open up http://localhost:37265 in a browser you should see your app presented to you by Symbiote, the app inspector embedded within Frank. That means you’ve successfully Frankified your application. Congratulations.

Write your first test

This first test will be based on the example that came with Frank from github (Frank Github repo). Please refer to it when trying to running these steps.

Step 1. Enable Accessibility Features

In OSX, go to System Preferences, Universal Access and check “Enable access for assistive devices”.

In the simulator open the Settings, then General->Accessibility->Accessibility Inspector and switch it to “On”.

Step 2. Build and Run

Select “Build->Build and Run” to launch the app. You will see lots of warnings as the libraries that Frank uses are compiled, these are safe to ignore.

You will see the app launch in the simulator. You may get a “Allow incoming connections” dialog, you should select “Allow” to let network connections in to the simulator

Confirm that everything is working by connecting to the simulator using a browser. Open http://localhost:37265/

Step 3. Write your first step

First create a subdirectory inside of features called step_definitions. This is where your custom step definitions will live.

When you ran cucumber if should have given you a sample step definition in yellow text. Copy the step definition that it shows, starting with “Given”.

In your editor, create a new file in your step_definitons directory called tutorial_steps.rb and paste in Given step definition

Delete the line that says “pending” and replace it with the following:

launch_app_in_simulator
wait_for_frank_to_come_up

So your whole step_definitions file should look like so:

Given /^the app is launched$/ do
   launch_app_in_simulator
   wait_for_frank_to_come_up
end

Run “cucumber tutorial.feature” now and the simulator should launch and everything will run green

You’ll see the words “FRANK!” get spelled out too in the output as the cucumber script connects to the frank server running in the simulator

Step 4. Write a second step to touch the plus button

Switch back to the tutorial.feature file and add a “When” line to specify the action being taken like so:

When I touch the Plus button

Run cucumber and copy the step snippet starting with “When /^I touch the Plus button$/ do” into your tutorial_steps.rb file

Now we need to figure out the UIScript that we need to send to actually touch the plus button. First we need to find out what we can about the button.

Start up a browser and navigate to “http://localhost:37265”
In the Selector entry field that comes up, type “navigationButton” (no quotes) and press * the “Flash matching elements link”. You should see the + and the Edit button outlines flash briefly.
Click on “Dump current DOM”
Search for “NavigationButton”, you should find a line with “class: UINavigationButton”
Looking at the DOM for this navigation button, we can see a few lines up that the accessibilityLabel has a value of “Edit”, so this is the first button

Go to the next “NavigationButton” in the search and you can see that the next one has an accessibilityLabel of “Add”

Lets make sure we’ve got the right button. Go back to the “Selector” entry field and type “navigationButton marked:’Add'” and then click the “Flash matching elements” link. You will see the “+” button outline flash.

Note: UISpec is sensitive about spaces and even leaving a space between the colon and the ‘Add’ will cause it to crash. No problem, just restart the Simulator.

Switch back to your tutorial_steps.rb file and change the pending line to be:

touch( "navigationButton marked:'Add'" )

Run cucumber and you should see a timestamp get added!

Your tutorial.feature file so far:

Feature: Drive our SampleNavApp using Cucumber
Scenario: Plus button adds timestamp
Given the app is launched
When I touch the Plus button
Your tutorial_steps.rb file so far:
Given /^the app is launched$/ do
launch_app_in_simulator
wait_for_frank_to_come_up
end
When /^I touch the Plus button$/ do
touch( "navigationButton marked:'Add'" )
end

At this point you could do a little refactoring to make the “Plus” a parameter

Step 5. Write a third step to validate the results

Add a line to the scenario in the tutorial.feature file:

Then I should see a table containing a timestamp
Run cucumber to get the step snippet and add it to the tutorial_steps.rb file

Switch to the browser and in the Selector field, type: “tableView” and click the “Flash matching elements” link. You will see the whole table flash

Now try entering “tableView tableViewCell first” (right out of the UISpec tutorial from here: ). You will see the first cell flash

Switch back to your tutorial_steps.rb and put in the following line:

cell_label = frankly_map( "tableView tableViewCell first", "text" );

This sends the selector to Frank for the first cell and then asks Frank to return its “text” attribute. In this case we get back the complete timestamp.

References

The links below provide background information that may come in handy as you are preparing to use Frank for the first time.

Sabin Pilipautanu

Sabin Pilipautanu

Engineering Manager

Sabin Pilipautanu is an Engineering Manager in 3Pillar’s Cluj, Romania office, and the head of our quality assurance competency center. As Engineering Manager, Sabin is responsible for collaborating with all members of our software development teams, including software architects, developers, and quality assurance testers. He works with our clients to help formulate and plan major software development initiatives into version release plans then oversees development of those development projects in an agile development environment. As head of 3Pillar’s quality assurance competency center, Sabin is responsible for establishing corporate best practices for software testing across a variety of platforms. This includes implementing best practices and conducting training for test strategy, test planning, mobile testing, test case design and reporting, and automated testing. Sabin has a BS in Mathematics from Liceul Teoretic Ion Luca Vatra Dornei in Romania.

Leave a Reply

Related Posts

The 3 Keys to Building Products That Drive Retention –... I had the privilege of being invited to speak at the Wearable Technology Show in Santa Clara this week, where I gave a bit of a reprisal of a talk I d...
High Availability and Automatic Failover in Hadoop Hadoop in Brief Hadoop is one of the most popular sets of big data processing technologies/frameworks in use today. From Adobe and eBay to Facebook a...
3Pillar CEO David DeWolf Quoted in Enterprise Mobility Excha... David DeWolf, Founder and CEO of 3Pillar Global, was recently quoted in a report by Enterprise Mobility Exchange on the necessity of understanding and...
How the Right Tech Stack Fuels Innovation – The Innova... On this episode of The Innovation Engine podcast, we take a look at how choosing the right tech stack can fuel innovation in your company. We'll talk ...
The Road to AWS re:Invent 2018 – Weekly Predictions, P... For the last two weeks, I’ve been making predictions of what might be announced at AWS’ upcoming re:Invent conference. In week 1, I made some guesses ...