The Essentials of Mobile App Testing and Monitoring

MobilePundits
www.mobilepundits.com

Introduction: You’ve developed a Mobile App...Now What?
By the time you finish reading this, the mobile app landscape will have changed. Along with the
development, testing mobile apps is quiet more challenging than testing based on desktop or web
applications.
Mobile app testing is an approach towards solving the emerging technological implications of
Mobile Apps. iOS and Android have been the most successful operating systems so far. It is
quite challenging to test and verify each and every application before releasing it for public use.
In today’s business environment, testing seems to be a cost-effective solution to avoid any
compromise on quality.
QA Challenges in Mobile Application Testing
Variation in Device
Mobile app testing becomes difficult because of many operating systems and myriad devices.
Compatibility is a major issue here because a mobile application can be deployed across multiple
devices. They differ in:
• Mobile operating systems like iOS, Android, BlackBerry, Windows, Symbian, etc.
• Different versions of these operating systems such as iOS 4.x, 5.x, BB 4.x, 5.x, 6.x, Android
2.0, 2.x, 3.x, 4.x, Windows 7, Windows 8.
Mobile Testing Tool Availability
The tools used for desktop and web-based applications do not work for mobile application.
Hence, a complex scripting technique and new tool development is required for mobile
application testing.

Industry Standards
Testing of mobile application must be performed according to certain industry standards so that
the applications can be accepted globally by people using different mobile operating systems.
This will help in making the application popular.
Necessity of Skilled QA specialist in Automation Testing
Every Quality team needs skilled quality assurance specialist not only to evaluate the automation
tools that are present in the market for application testing but also to quickly recognize any bugs
in the user interface as UI is very critical in the mobile world. A well-examined and fully
functional mobile application can be easily be rejected by end users because of bad interface
even though if it is working perfectly.

Cloud Computing
This technology offers a seamless web-based environment for mobile on simulators where the
QA team can deploy, automate and test any mobile application. This approach of cloud
computing has really helped in reducing the overall cost of a project under development.
Listed below are some of the most common problems that are faced by the testers in an
environment of cloud computing:
•High-cost
•A Subscription Model
•Lock-in
•The Automation is time-consuming and image based
•Automation cannot be done outside the framework
Mobile Testing Challenges for Native Applications
For many, “mobile application” means a native or a hybrid application. These are some
commonly downloaded software applications from the respective app stores. The unique
experience that they offer to the users improves the capabilities of the operating system and
device for which they are developed. The downloaded is often monitored by the store, with
mechanisms in place to charge consumers.

Though native applications offer rich user experience to the users – and plausible money
spinning for the developers. Some complexity is added to the lives of those who test it. Testing is
meant to make sure that the application can be successfully downloaded and executed on the
device. To meet the challenges of the testing procedure, backward compatibility with each older
device you are expected to support must also be checked.
Mobile Testing Challenges for Web Applications
Like any other website on internet, a mobile web app is also viewed by people around the world.
Even when you are targeting people from one particular landscape or a network, it will help you
understand dynamics of the world.

Mobile Application Testing Strategy
The Test strategy should make sure that all the quality and performance guidelines are met. A
few pointers in this area:
Selection of the devices
Analyze the market and choose the devices that are widely used. (This decision mostly relies on
the clients. The client or the app builders consider the popularity factor of a certain devices as
well as the marketing needs for the application to decide what handsets to use for testing.)
Emulators
The use of emulators is extremely useful in the initial stages of development, as they allow quick
and efficient checking of the app. Emulator is a system that runs software from one environment
to another environment without changing the software itself. It duplicates the features and work
on real system.
Types of Mobile Emulators
 Device Emulator- provided by device manufacturers
 Browser Emulator- simulates mobile browser environments.
 Operating systems Emulator- Apple provides emulators for iPhones, Microsoft for
Windows phones and Google Android phones
Some free and easy to use mobile device emulators are:
iPhone Tester – All you need to do with this is – enter the URL in search box and you can see
the real time preview of how it appears on an iPhone.

Mobile Phone Emulator – Used to test handsets like iPhone, blackberry, HTC, Samsung etc.
MobiReady– With this, not only can we test the web app, we can also check the code.
Responsivepx– It checks the responses of the web pages, appearances and functionality of the
websites.

Screenfly– It is a customizable tool and used to test websites under different categories.
Mobile cloud Computing Services
The testing experts can efficiently make use of the environment of mobile cloud computing for
deploying and testing the application. Using this technique, we can keep both the task and the
data on the internet instead of keeping it on an individual device. This is what brings on-demand
access in vogue. Lowering down the complexity of project implementation, organizations take
the advantage of services of cloud computing so as to reduce the development and testing cost
and boost the return on investment of a mobile software application solution.

Real Times Devices with Real Networks
It is quite natural that testing can be done against real networks. Real devices can be used at real
locations for performing testing. This is something that the QA team just cannot avoid. Testing
of real devices on real networks should always be an option whenever needed. This thing came
into picture because every application will be used at different locations by end users who might
use it from a remote area with inconsistent network signal strength.
Avoid Manual Work with Automation Tools

Automate whenever and wherever possible! Remote and Emulators, real time devices and
solutions which support playback functionality and scripts help in saving and allow execution of
more test cases with higher degree of consistency.
Any testing team must automate the task of mobile software applications so as to avoid any
manual task. This will not only save cost but time as well. There are many tools available in the
market for performing automation tools.
Types of Testing for a Mobile Application
Usability Testing
This is the type of testing that includes visibility of text in the selected language, screen
navigation, verification of online and offline functionality, feedback from interaction with the
system, which means that the download app should be prompt with messages.

Compatibility Testing
Compatibility testing means validation of mobile application for myriad devices keeping in mind
all the major operating systems, screen size and resolution as per the demand of users,
proofreading if the integration server changes, editing of the mobile app isolation with other
software apps on the device.
Interface Testing
Following are the things that integration testing covers:
•Screens Resolution
•Buttons
•Text Inputs
•Navigation flow like Facebook, Twitter, Reviews, Bookmarks, etc.

Service Testing
Validation of mobile software application not to behave as a server is included in this type of
testing. It even covers checking of a service which can be used offline or takes too long or when
a service goes down and responds back with malformed responses.
Performance Testing
Performance testing is something where you will need to verify the server connection of WIFI
from 2G/3G or vice versa, size of the shared images used by the software code is in accordance
with the desired response time, optimization of code with the CPU cycle, consumption of
battery, memory leaks, certain resources like camera, GPS, etc., freed.

Operational Testing
This type of testing checks for the back-up of all the important information in the application,
save and recovery plan if the battery goes down, if the data of mobile app is lost in the case of
up-gradation from respective app store, accessibility of the app when user gets an alarm,
message, call, reminder, etc. and power usage while accessing the application.
Low Level Resource Testing
Overuse of memory and not releasing is something that comes under low level resource testing.
It also checks if temporary files of the applications are not cleaned, local database is growing too
large. Low level testing will also see the garbage generated by the mobile app.

Security Testing
Security testing will include the encryption and decryption techniques used for communication
of sensitive data, checking of multi-user support without any interference of the data between
them, and checking of accessibility to files that are saved in the app by any unintended users and
detect areas where tested application does not receive any nasty content.
Testing Recommendation
For the types of testing mentioned it is a good idea to use some combination of real device and
emulator testing, as recommended in the table below:

Enterprise applications must meet requirements for maintenance, administration and security.
They “are more complex in functionality and architecture” and therefore it is important to test
enterprise application on performance, security, and synchronization in addition to the standard
functionality testing.
Test Automation Tools & Frameworks
Testing mobile applications is traditionally done by manual execution of test cases and visual
verification of the results. However, it is very time consuming. Using automation tools and
testing frameworks yields quantifiable benefits and is recommended.
Automated testing offers fast, repeatable and comprehensive test execution, including overnight
or over weekend test runs. However the investment in tools, test script development and data
collection is required and is and is not amenable to fast changing requirements.
The development platform for mobile application which is a Personal Computer is different from
the target platform which is a handheld device. It is impossible to test the various possible states
a GUI can have. Because of these unlimited testing scenarios mobile GUI testing is more
difficult compared to functional testing of desktop application.“Manual GUI testing is very error
prone and hardly reproducible, and causes very high effort”.
Android Testing Framework
The Android testing framework provides powerful tools for testing mobile applications. The
following diagram summarizes the testing framework of Android:

The Android SDK (Software Development Kit) consists of tools for developing and testing
android based mobile application. The Android SDK tools are available as Plug-in in Eclipse
with ADT, and “in command-line form for use with other IDES”. The SDK“tools get
information from the project of the application under test and use this information to
automatically create the build files, manifest file, and directory structure for the test package”.
The SDK also provides monkeyRunner, an API testing devices with Python programs, and
UI/Application Exerciser Monkey, a command-line tool for stress-testing UIs by sending
pseudo-random events to a device.
Sikuli
Sikuli Test is a GUI testing framework that enables automation testing tasks.“It allows testers to
write visual scripts to automate tests, to refer to GUI objects by their visual representation
directly, and to provide robustness to changes in spatial arrangements of GUI components”. The
script uses action statements to simulate the interactions and assertion statements to visually
verify the outcomes of the
interactions.
Test scripts under Sikuli Test are written to test traditional desktop GUI applications on
Windows and Mac OSX, as well as mobile applications in an Android emulator and iOs
simulators.

JaBUTi/ME: White Box Testing
White box testing is a technique based on the internal structure of a given implementation, from
which the test requirements are derived. In general, “white box testing criteria use a
representation known as Control Flow Graph (CFG) to abstract the structure of the program or of
part of the program, as a procedure or method”.
Server based testing JaBUTi/ME
JaBUTi (Java Bytecode Understanding and Testing Tool) is a complete tool suite for
understanding and testing Java programs and Java-based components. JaBUTi differs “from
other testing tools because it performs the static and analysis directly on Java Bytecode not on
the Java source code”.
Mobile Test: Black Box Testing

An automatic black box testing tool for mobile devices introduced in this Chapter is Mobile Test.
With Mobile Test building maintainable and reusable test cases for testing system level and
application level software on various mobile devices is possible. Mobile Test observes the input
and output information of an application. “From the input perspective, a mobile application
receives two kinds of inputs“. The first input is form the user GUI such as keyboard events,
touch events. The second input is environmental context events.
Mobile Test Architecture
Mobile Test architecture
The architecture of Mobile Test subdivides the system environment into layers to reduce the
complexity the system. Each layer provides services to upper layers with the support of lower
layers. In this way, the test control layer can be separated from the characteristics of the
underlying devices.

List of available testing tools
The table below gives a list of available testing frameworks with the platforms they can be tested
on.
Available mobile automation testing tools and the platforms they run on

Fledge: BlackBerry device simulator that enables mobile application testing on a Personal
Computer. Tests, various connectivity and state changes can be simulated.
MonkeyTalk (formerly known as FoneMonkey): offers the possibilities to save a script, load,
read and modify it. FoneMonkey is designed to support developers and quality control tests.
FoneMonkey automates testing on iOS simulators, Android emulators or real devices.
QTP(QuickTest Professional): is an automated testing tool provided by HP/Mercury
Interactive. QTP uses VB scripting language to build its flows. It provides automated and
regression testing and also generates test scripts that can be executed on local or remote mobile
devices.
CTS (Compatibility test Suit): runs on Personal Computer and manages test execution.
Individual test cases are executed on attached mobile devices or on an emulator.
UISpec: Behavior Driven Development framework for the iPhone that provides a full automated
testing solution that drives the actual iPhone UI. It is modeled after the very popular RSpec for
Ruby.
Hermes: black box automation tool for testing J2ME applications. Hermes supports
application independence, and open interfaces for extensibility.
Eggplant: black box automation tool that runs on Mac OSX and Linux. It can test applications
on a vast range of other platforms because it connects to and controls them using VNC (Virtual
Network Computing).
Robotium: black box automation tool for Android Mobile Application. Robotium is a UI testing
tool. The Framework provides API’s to test various kinds of Widgets/UI present in mobile
applications developed with android SDK.

Conclusion
Application development for mobile devices is evolving. The strategies presented in this thesis
discuss the testing of mobile device applications and how important it is to plan a test strategy
that is mobile-specific. Unique challenges like device challenges and software challenges of
mobile devices need to be considered because traditional testing does not cover all characteristic
important for mobile application. The use of automation test tools and test methods should be
conducted for a successful testing result.
The advantage of testing these apps before the deployment in a live environment cannot be
stressed out. An app with bugs and errors will not only hurt the productivity but reputation as
well.
About MobilePundits
MobilePundits partners with global enterprises to drive their innovation-led growth. That's why Deloitte awarded MobilePundits
among the top 100 most innovative companies. As a leading provider of next-generation consulting, technology and outsourcing
solutions, MobilePundits helps clients across the world. Visit www.mobilepundits.com and see how MobilePundits, offers its best
outsource and development services.
For more information, contact marketing@mobilepundits.com

The Essentials of Mobile App Testing and Monitoring

More Related Content

What's hot

Viewers also liked

Similar to The Essentials of Mobile App Testing and Monitoring

More from MobilePundits

Recently uploaded

The Essentials of Mobile App Testing and Monitoring