Mobile Automated Testing With Perfecto Mobile Part i

Introduction

The Perfecto MobileCloud is a completely web based Software as a Service (SaaS) platform that allows mobile application developers and QA Engineers  to work with services like advanced automation, monitoring and testing services. It is the right tool in the hands of a mobile app tester to automate testing for mobile phone devices connected with diverse mobile networks spread across different geo-locations.

The automated testing is done using an intuitive web-based user interface design, which allows the creation of tests scripts which are easy to generate and easy to use with no coding involved at all. The commands comprising the scripts are readily available on the websites in the form of widgets. Test developers just need to click on the commands to add to the scripts and define its properties in the user interface. Keyword based Script Once technology enables QA Engineers to create test scripts. Perfecto Mobile testing tool works on image or text recognition. The generation of these scripts can be watched in the following videos.

Test developers can easily install third Party applications or APK files on the remote devices using their Install widget. Upon deployment, the application will be visible in the Application list of the device.

The automation tools work on the basis of OCR based text and image recognition mechanism. This enables the tool to analyze and test the applications’s user interface without any limitation. From native apps, web based apps, or hybrid apps tools it can test anything. As far as Flash content is considered, the sales team are not sure whether the tools can test this or not.

For both the tools, there exists technical support if we require them.

Perfecto Mobile Cloud provides a Hybrid objects approach, which consists of both Visual (OCR) based objects and native objects for iOS/Android applications, as well as native DOM objects for Web/Hybrid based applications. It offers both HP UFT or a native automation framework, an object spy tool which allows easy object filtering and identification and easy scripting.

They also provide 3 different cloud based deployment options as part of their offering:
Public cloud, Private cloud (either hosted or on-premise) and Local solution (within the customer local LAN network) – called MobileCloud Express.

Early in the mobile app development process it usually makes sense to conduct manual testing while the test strategy is being refined. Then once the testing processes are standardized, automated testing starts to deliver significant benefits.

Perfecto Mobile Cloud

The user license with Perfecto Mobile offers these variations:

1. Perfecto Mobile Cloud Interactive
Using Perfecto MobileCloud Interactive services, a user can quite easily test websites, services or mobile applications even manually. They can hold the device in their hands and perform testing on multitude of mobile handsets and devices  connected to live networks using the web. The devices operates seamlessly in all the major mobile networks across various countries. It has active mobile data plans and enables you to make/receive calls, send/accept messages, install your apps and browse the Internet. The free trial for this can be taken up for 1 hour. You can take view of how to interact with the devices:

Perfecto MobileCloud Interactive services

 

  • Users can access logs generated by the remote devices through the Device Log widget available on the website.
  • Enables access to various other functionality which captures internal performance of the device like, Battery Level, Battery voltage, CPU usage, Memory Usage, Memory Free, Bytes In and Bytes out for Mobile/WLAN of the device.
  • Enables to share live session of the device with other users to demo new apps/features, reproduce a bug. It allows me to just share the session URL or invite other participants. The session would be secured. Stopping the session would end the sharing mode.
  • Just before releasing the device, users can execute Clean Up widget on the device to make sure that application is uninstalled from the device.
  • The widgets on the website enables the users to change orientation view of the application, change the orientation of the device, zoom in/out the view of the device screen, position the device on the screen etc.
  • Users can Reserve a particular device for said period of time (paid feature).
  • The real handset devices have SIM card inserted in them and so the Device panel shows the carrier information, network information, phone number, location and many other information of the SIM card. To verify this, I called the phone number of Motorola Razr as available on their website which is reflected on the mobile screen:
Perfecto Mobile Cloud

2. Perfecto Mobile Cloud Automation and Mobile Cloud Reporting
MobileCloud-Automation allows us to execute automated testing on real handsets and tablets located anywhere in the world.

  • The test scripts are written using keyword-based Script Once Technology. The creation of scripts are easy.
  • The scripting technology is keyword driven and business requirements driven test scripts.
  • The generated test scripts are highly device agnostic. Same test scripts can be executed across all mobile handsets irrespective of OS, platform or manufacturer of the device without any modification. However, specific test cases can be defined for specific devices.
  • The test script generates detailed report of the executed test script at the end with vast variety of information such as actual results, screen captures in digital format, continuous video recording of the entire script and graphs showing statistics about the execution.
  • The test report can be shared with other users and can also be exported to HTML, XML, PDF, CSV format.
  • The test script can perform:
    • Functional Testing
    • User Interface Testing: In this, the script is able to capture UI distortions like truncated/overlapping texts. It can test alignment of views to each other and their respective placement on the screen. The screen layout can define relationship between the views (above /below), spacing between them (in percentage) etc.
    • Performance Testing: In this tool, there exists full control on the flow of the application like conditions, loops, flow commands, expressions and data-tables. While writing the test scripts, set of user actions can be written in the loop. These set of actions would comprise of all the activities which we want to test against its performance.
    • Testing of Web based content or Hybrid App: The automated test script scans/analyzes the mobile screens and works on Text and Image based recognition (OCR) mechanism. So the application can either be native, browser app or hybrid app, the test script analyzes the components placed on the screen. This includes testing of two different package applications in one test script (example, native application invoking browser application).
  • Data driven Testing approach can be followed in these devices. The mobile cloud computing supports creation of data tables in the private repository and can also import the data through csv file format.

Among hardware features:

  1. Camera is accessible to the users but the view behind the camera is grayed out. In this aspect, some camera based functionality such as taking snapshot of the vicinity or barcode scanning through camera would not be working. According to the sales team of Perfecto Mobile Cloud, they would provide an API, in near future, to inject an image to the camera API so that mobile apps can be tested.
  2. GPS based APIs work on these devices. Once the GPS is turned on, APIs based on this technology would work. According to the Perfecto Mobile Cloud sales team, the tool facilitates to set the direction to the GPS/maps.

Video1: Perfecto Mobile – Getting Started

Video 2: Perfecto Mobile – Mobile Automated Testing

 

3. Perfecto Mobile Cloud Monitoring

Provides 24 by 7 user experience design monitoring of mobile services and Real Handsets and Tablets.

Although mobile automation tool would not eradicate the manual testing process, there exists few reasons to automate testing of smartphones native applications:

  1. It saves time and effort. Repetitive testing on each iteration would be now be less tedious. This would also allow QA to focus on quality.
  2. It would expand test case coverage in the application and finally release reliable/robust application. In all, improving the app quality.
  3. Same test case scripts can be executed on different mobiles irrespective of operating systems and language it supports. This covers a wide range of devices (Android, iOS, Blackberry etc) on which automated testing can be performed in very less time.
  4. Improves effectiveness of QA testing process.

Following are the mobile phone functionalities which were tested on Android phones while creating the automated test scripts:

1. User Interface controls and actions performed on them:

  • Button Click
  • Radio Button Click
  • TextView Click
  • Image Click
  • Checkbox Click
  • Text Input In Edittext
  • Text Input to Edittext when there are multiple Edittext present on the screen (since the test script does not work on Ids).
  • Listview Row Click
  • Button Click on AlertDialog

2. Automated Scrolling of the Webview or Listview UI component and performing the actions mentioned in the Test Script.

3. Validates and verifies Image displayed in the application (either in native application or in web page).

4. Validates and verifies the text anywhere in the application including Alert dialogs/Activities/other UI components and compares with the expected string value.

5. Two or more applications can be tested in one test script. This also implies to testing of native and web applications in one test script.

6. Search mechanism can be implemented in the current screen of the application. The tool can search for an image or texts mentioned in the test script in the displayed Activity of the application and can perform the actions on them.

7. Wait mechanism can be implemented in the test script. This can be for a specified period of time (let’s say 15 seconds) or until that a Text/Image appears on the screen (Text Sync, Image Sync, and Image Checkpoint).

8. Captures and validates UI distortions like:

a.        Overlapping texts/images

b.       Truncated texts/images

c.        Misplaced texts/Images

d.       Alignment of UI elements to each other ( to a certain extend).

9. Conditional testing on actions can be implemented. Based on state of the variable/application/UI component, further testing can be chosen. If/ Else commands can be added.

10. Looping command can be added in the test script. A set of actions can be executed in loops.

11. Reports can be generated in various formats like HTML, XML, PDF etc. These reports can be downloaded or saved for future reference. The test execution can be recorded and played afterwards. These can also be shared on mails in the form of videos.

12. Maintains repository online which is helpful in storing Images/Scripts/XML etc. for easy retrieval.

13. Positioning of UI elements can be tested on relationship basis. Each UI element can be tested whether it is placed at Top/Bottom/Left/Right of other UI element.

14. Automated Launch/Close/Install/Uninstall of a particular application can be done.

15. Script Actions can be commented or excluded for later execution.

16. Camera on mobiles is accessible but taking a snapshot from it is not useful.

17. Hardware keys works: Back Key, Menu key, Volume Up/down, Power off.

18. Error handling mechanism is supported in the text script: catch/break/continue etc.

19. Data driven testing can be done. The tool supports creation of database online or import the database from local drive.

iPhone Web Browser Automated Testing:

Web browser controls and actions performed on them:

1.    Launching of web browser

2.    Selecting address bar of the browser

3.    Navigating to a particular browser

4.    Text input in text field.

5.    Click on an image

6.    Click on hyperlink

7.    Waiting for the web page to load

8.    Moving Back and forward on web browser

9.    Handling the alert shown in between. Selecting Ok /Cancel on Alert.

A robust software testing methodology must weigh the pros and cons of going ahead with a mobile automation approach. Many a times, creation of auto tests can be a time consuming exercise in comparison to the traditional approach of manual mobile testing. For Quality Assurance Software Engineers, it is the flexibility and convenience of analyzing results that often sways the balance in the favor of mobile automated testing.

Khushboo Kaur

Khushboo Kaur

Software Engineer II

Khushboo Kaur is a Software Engineer II at 3Pillar Global. Her responsibility in the company is to develop Android Mobile Applications. She is a Sun Certified Java Professional and has proficiency in Java Language ranging from web applications, standalone desktop applications to mobile applications on Java and Android mobile operating systems. She has worked on projects like CSR (bluetooth and music based app), LiveGuide (ticketing and barcode scanning application) and has experience on Mobile Automation. Prior to joining 3Pillar, Khushboo has worked with one of India’s largest VAS companies in the Telecom sector where she has developed several mobile applications .

7 Responses to “Mobile Automated Testing With Perfecto Mobile Part i”
  1. Deepak Elugoti on

    Very interesting tool, can you please share me the link to download a free trial software link.

    Reply
  2. Rajasekaran on

    Hi,
    The article was very useful..however at my current project am using selenium mobile-drivers for hybrid app testing. In this whatever the identifiers i give am unable to identify the app. Kindly share if u have any info on how do we proceed with that.
    Kindly help me to get the below given details in the code from developer for the given .Apk file
    nativeDriver.open(applicationPackage, applicationActivity, MobileintentOptions);

    Thanks In Advance,
    Rajasekaran N
    +91-9952992387

    Reply
  3. Dhilip Prasanna on

    Excellent article regarding mobile automation. It is written in a simple manner with neat content and presentation. Thank You.

    Reply
  4. Alok on

    Thank you for introducing Perfecto , however please have a look on my below queries
    1: Can we record the various activities like we do in SOASTA and Loadrunner ?
    2: Can we perform the testing from various geographical location ,I mean can we have mobile data instead of WIFI as i have seen the demo video ?
    3:Can we have Mobile Gestures like Zoom in ,out,Multiple Touch ,Termination of App etc?
    4:Can we have Background Image Validation ,Validation with Appropriate Pixels ,Validation of Various Text and Hyper Link ?
    5: if i have created the one script for any mobile app and after some new build if any UI changes in APP ,do I need to develop the new script or I can reuse the existing clip and edit it ?

    Thanks,
    Alok

    Reply
  5. Sanchit on

    How can we get the X path of Drop down in IOS.
    I have 10 items in drop down i want to select 9th item from the drop down.
    but i can see only 3 items in screen rest others are hidden.
    Kindly help with it.

    Reply
  6. ravi on

    i am using perfecto but it is not showing few of the devices.

    Reply
  7. james on

    Hi,

    I am using pcloudy (www.pcloudy.com), is it good to use for mobile app testing ?

    Reply
Leave a Reply

Related Posts

The 3 Keys to Building Products That Drive Retention –... I had the privilege of being invited to speak at the Wearable Technology Show in Santa Clara this week, where I gave a bit of a reprisal of a talk I d...
High Availability and Automatic Failover in Hadoop Hadoop in Brief Hadoop is one of the most popular sets of big data processing technologies/frameworks in use today. From Adobe and eBay to Facebook a...
3Pillar CEO David DeWolf Quoted in Enterprise Mobility Excha... David DeWolf, Founder and CEO of 3Pillar Global, was recently quoted in a report by Enterprise Mobility Exchange on the necessity of understanding and...
How the Right Tech Stack Fuels Innovation – The Innova... On this episode of The Innovation Engine podcast, we take a look at how choosing the right tech stack can fuel innovation in your company. We'll talk ...
The Road to AWS re:Invent 2018 – Weekly Predictions, P... For the last two weeks, I’ve been making predictions of what might be announced at AWS’ upcoming re:Invent conference. In week 1, I made some guesses ...