October 26, 2017
How to Overcome Challenges Faced in Hybrid App Testing
The mobile boom is self-evident, and it is here to stay. Smartphones are rapidly becoming the primary means of interaction for consumers and businesses worldwide, with thousands of apps created each day. Mobile goes beyond smartphones and tablets. Today, apps are incorporated into cars, wearable tech, and home appliances.
Hybrid apps are being adopted by people of all age groups across industry verticals. Most applications today, including famous ones like Spotify, Slack, Skype, and others, are all hybrid. However, hybrid apps pose some unique challenges when it comes to QA. Here, I will be giving insights into how the nature of hybrid apps requires an additional consideration during the testing strategy.
What are Hybrid apps, exactly?
With a native app, you have either an iOS code or an Android code, and you have to write code either for one of them or two sets of code. However, if you are using a hybrid framework, what you could do is build your code on top of that code, and the framework will translate everything into two different platforms. Let's say, an iPhone takes a picture in a certain way, and Android takes a picture in a different way using two sets of codes. This framework will consider that and will give you one standard set of codes.
Challenges In Hybrid App Development
For native apps, the scope of testing is confined to either iOS or Android, but in the case of Hybrid apps, the extent of testing gets wider. Based on my experience, I highlighted the most common issues mobile application testers face daily. However, many other challenges can be case-specific, and I did not include them on my list.
A Hybrid app needs to be tested on two different platforms:
For Android, the OS-versions we tested was from version 4.4 to the latest version. For iOS, we tested upwards from version 8.0.
For different platforms, testing is required for the following:
Different OS Versions
We sometimes find issues that are specific to one OS version. For example, in my last project, I faced a problem where in one module, the text was truncated at the bottom in iOS8, whereas in iOS9 it was working fine. It was therefore required to be verified on all the different platforms available.
Testing your app on various screen resolutions
The layout of the app needs to be checked on different screen resolutions. This is necessary because we may have a significant number of screen resolutions available in the market for one mobile OS (like Android).
UI/UX testing for hybrid applications involves testing across various devices and operating systems, as all devices carry the conventions that their users expect. This approach ensures that the UI elements, like toolbar, menus, images, lists, and edit fields, in the hybrid application work consistently across multiple target platforms and provide a better usability experience.
Testing your app in varied region and time format settings
The application sometimes has different functionality depending on region and time format settings. For example, changing the region and time format of the device will also modify the time format in your app.
Testing the app with other third-party applications
Some users install third-party apps that are accessible in your hybrid application. Sometimes these third-party apps can crash your application. Therefore, hybrid apps are required to be tested with third-party apps that may be used within your application. For example, when we tried using an app we developed with the Swype keyboard, the app crashed.
Testing your App in various Network condition
Many applications today are network dependent, so the app needs to be tested on different network conditions like signal loss and bandwidth variations. The variability of a network can have different impacts on the application; for example, unpredictable application behavior, user interface related errors, database corruption, performance challenges and functional issues.
To test applications under different network conditions, I throttled the connection using Charles Proxy tool. Charles adjusts the bandwidth and latency of your Internet connection.
Automation and test case writing
Hybrid apps can make automation both easy and hard. If it's built only on web-based languages and adapted with as little native coding as possible, it is effortless for QA engineers to write automated scripts that function across various OSes, which is indeed a novelty. However, if the app is created with entirely different languages and has very different navigation features, then writing new automated scripts is required for each supported platform.
Beyond functional testing, performance must be assessed on the individual device level as well as from a usability and UX perspective. The performance of a hybrid app is moderate compared to native apps. Therefore, certain features in your app may take a little extra time to respond in comparison to native apps.
One of the advantages with hybrid applications is that development and maintenance can be more straightforward because one fix in the web could mend an issue in all the supported platforms. However, it is not at all easy for testing. This is because even if a fix or newly developed feature is just in the web code, you cannot take for granted that it will work the same on all the platforms. We saw many cases where a fix would work on iOS but not on Android, or worked on iOS 9.0 but not on iOS 8.1. So for each small fix in the web, it still needs to be tested on a large number of devices.
Because of this, I started to look at some automation frameworks and started using Appium, which is a cross-platform test framework that has support for Android, iOS and Windows Phone.
Key takeaways from developing Hybrid apps
- Hybrid apps take less time to develop. If you have less than four months to develop an app, and you want to test a limited private market on the viability of your app, then use Hybrid.
- Hybrid applications provide cross-platform compatibility, with an emphasis on accessibility to native features, which results in good user experience.
- Hybrid apps have a lower total cost of ownership owing to easier upgrades, lower maintenance costs, and development using familiar tools and technologies.
- Hybrid apps provide access to native features like camera, GPS, Bluetooth, contacts, etc.
- You can use your existing web talent and don’t need to bring on additional resources.
- Hidden bug potential – Hybrid apps add an extra layer to counter any bugs that may exist. Frameworks themselves can sometimes have bugs.
- Hybrid apps can be made available and distributed via a relevant app store, just like native apps.
- Hybrid apps appear to the end user as native apps but run significantly slower than native apps.
Justification for Hybrid apps
|Multi-platform Support||Hybrid apps can be deployed on multiple platforms. Although, for each platform, a particular native module/container must be developed.|
|Push Notifications||Real-time push notifications are possible with the hybrid approach via the use of native components. HTML5 also supports notifications via server push technology with persistent connections.|
|Database Storage||In hybrid applications, data can be stored securely with encryption.|
|Response Time||In terms of performance, hybrid apps are slower than native apps.|
|Development Cost||the development of hybrid apps is cheaper than native apps because the hybrid apps development process involves writing a native shell/container and wrapper over the existing presentation layer using third-party libraries.|
|Code Reuse and Duplication||Since platform-specific code is required to be written to build native apps, there is often a good deal of code duplication. Each platform-specific code needs to be rewritten in its OS-supported language. A hybrid app can leverage standard server-side components because the same HTML content is exposed for each platform.|
|Cost/Time-to Market||Since Hybrid apps can address multiple platforms better and require less time to develop and deploy, rolling out hybrid apps is less costly and time-consuming.|
|Resource Pool||Mobile-platform-specific developers specialized in iOS or Android are not as easily available as web developers.|
|Future Release/ Enhancements||With a hybrid application, rolling out new features or enhancements is much easier because no client app distribution is required.|
|Security||Typically, enterprise applications are exposed over the Web and consumed via hybrid applications using an HTTPS protocol, which means that no sensitive data is stored on a device. It reduces the risks of data theft or an application being exposed to potential hackers and malware in the user's device.|
Best Practices for Hybrid Testing
The following are best practices that should be kept in mind while testing a hybrid mobile app:
- Test the app as a Black Box and try to break it
- UI testing should focus on validating how the application handles the touch events, mouse events, keyboard strokes, and gestures against the custom UI guidelines.
- UI/UX features such as page transitions (with slide up, slide down), swipes (with a tap, tap hold, double tap, swipe left, swipe right features), and platform-specific look and feel of controls (like time picker) are required to be tested on different platforms.
- Performance Testing: It has been observed that 60% of users abandon a mobile application if it does not load within 3 seconds, so performance should be one of the critical concerns for mobile app developers. Hybrid applications are slower than native apps.
- Security Testing: If your app is writing sensitive information to the file system at any point, such as credentials, signature files, usernames, passwords, and API keys, authentication tokens should be verified on all the platforms.
- Don’t just restrict non-functional testing to performance or security. Carry out the following tests:
- Installation testing
- Uninstallation testing
- Idle state testing
- Offline mode testing
- Interprocess communication testing such as incoming call, text message, and other app notifications while using app
- Battery performance while using the app
- Performance of app in different network conditions
Beta Test App for Early Feedback
For early feedback, we can distribute the app internally through various tools available. Some tools to consider using include: Test Flight for an iOS app, Crashlytics for iOS and Android, and HockeyApp for iOS and Android.
To be Hybrid or not?
Below is a checklist that will help us figure out if hybrid apps are the best way forward given the requirements, context and end users.
- Hybrid app development may become popular for developers and companies that do not have a platform preference.
- We have many clients that want to develop relatively simple mobile apps for several mobile platforms at the same time, and they want to get these apps out there as quickly as possible, before their competitors.
If we look at statistics (such as the recent survey by RedHat), 50% of businesses today need mobile development specialists. However, what's most interesting is that among these firms, native app developers are not in high demand. As it turns out, front-end web developers and back-end integration specialists take care of a significant portion of mobile development tasks these days. I think this is a good indication that mobile development is shifting from native development towards hybrid app development.
While it is impossible to foresee every possible outcome, we can certainly make a few safe assertions. It is safe to assume that hybrid app development will continue to grow in popularity, for both consumers and developers.
Hybrid mobile apps are here to stay and are certainly a big part of the future of overall mobile applications development. It will be interesting to see the next few generations of hybrid mobile apps, and how they support the next generation of mobile devices as they emerge onto the consumer market.