default title

Background

With the widespread popularity of mobile devices, the market share of mobile devices such as smartphones, tablets, and digital TVs is increasing day by day; According to relevant reports, the current number of smartphone users has exceeded 6 billion, and it is expected to reach a new height of about 7.69 billion by 2027. With the rise of this number, the mobile application industry will also face new challenges, and the market will provide increasingly diverse and complex user experiences to meet user needs. To address this challenge, ensuring that mobile applications meet all standards and have no significant defects before entering the user market is a very important issue. The solution to this problem is to use appropriate mobile testing tools and well-defined strategies for effective mobile application testing.

As test engineers, how can we improve the delivery quality of mobile applications in the face of diverse and complex applications? Below, we provide some strategies for mobile testing around this issue for your reference.

What is mobile application testing?

Mobile application testing is the process of evaluating the functionality, performance, availability, and security of an application across various mobile devices, operating systems, and network environments. Its purpose is to ensure a high-quality, error free user experience and confirm that the application meets the predetermined goals and needs.

Reserve plan

1、 Clearly define the test object

Before we test and discuss mobile application testing strategies, we should first clarify the testing objects. Mobile applications are typically used on mobile devices such as smartphones, tablets, and digital TVs. And these mobile applications usually use the following development modes:

Native developed applications: These applications are designed to serve users on specific mobile platforms and are available on the Play Store or AppStore. This type of application is developed by using Objective C/Swift for iOS or Java/Kotlin for Android. For cross platform native applications, NativeScript, Flutter, or ReactNative are the most commonly used languages.

Hybrid applications: The functionality of hybrid applications is similar to that of native applications, with the difference being that these applications are web applications developed using network technology, typically running within native containers, presenting HTML using the device's browser, and processing JS locally.

Responsive web applications: These applications are built specifically for running on mobile browsers and are PWA or highly responsive versions of websites that provide additional user-friendly mobile features.

2、 Mainstream mobile testing solutions

Currently, the two main mobile testing solutions available in the market are manual and automatic mobile testing. Nowadays, the increasingly complex features and functions delivered through mobile devices make it challenging to solely rely on manual testing. Because QA experts perform end-to-end testing without automated tools, extensions, or scripts, which is often tedious and time-consuming, automated application testing can be effective. QA experts typically use mobile application testing tools to write appropriate scripts, execute these scripts, and obtain data. Mobile application testing automation is mainly used for testing functions, which requires simultaneous analysis of requirement logic and conversion into technical language, and writing into executable testing code.

3、 Testing method

a. Cross platform testing

Nowadays, users can freely choose various devices and operating systems to access applications. Therefore, perfect performance must be provided on these platforms. After determining the appropriate operating system version to support, it is crucial to identify which devices are the correct matches and conduct testing. Testers can combine simulators with real equipment testing to make equipment testing more cost-effective and achieve good test results.

b. Security testing

Security is one of the most concerning issues for users today. As mobile devices become an indispensable part of our lives, we often share large amounts of sensitive data such as identity information, passwords, location history, etc., making this data vulnerable to threats. Therefore, security testing must be performed to verify the application's resistance to malicious user attacks. In order to deploy effective security testing, it is necessary to implement a robust and well-defined strategy as the foundation for effectively bridging security vulnerabilities. When developing a security testing strategy, it is crucial to consider the following factors:

(1) Understand the environment or platform that will be used to run the application, and identify vectors that attackers may use on specific operating systems.

(2) Develop multiple lines of defense, including mobile application testing tools with static, dynamic, and forensic analysis, to simulate different ways attackers may invade applications.

(3) Running tests from the attacker's perspective enables users to better understand the program's weaknesses and traps, as well as how attackers can exploit them for intrusion.

c. Network connectivity testing

Although users may spend most of their time using office WiFi or have good network connections at home, a considerable number of people still face the challenges of unstable network connections and low-speed Internet. Under such unfavorable conditions, inconsistent network reception can result in high latency or poor performance when switching between variable network speeds. At this point, the QA team must ensure seamless and consistent performance across various network speeds and conditions, from 3G, 4G, 5G to LTE and WiFi, taking into account sudden changes in network and speed, and even performance resulting from disconnection.

d. User Experience/User Interface Testing

Typically, an excellent mobile application relies on its humanized and elegant user interface (UI) and user experience (UX). On the one hand, efficient UI testing helps explain the ease of use, consistency and logic, accessibility and compatibility of an application, making it easier to understand how the application interacts with end-users. On the other hand, user experience testing involves testing multiple aspects of user experience to determine the correct match between the application and its elements and audience interaction. When conducting UX testing, it is necessary to ensure the following checkpoints: easy navigation through the application, fast trading and ordering processes (if any), sufficient user access to product information, improved user experience based on user feedback, and so on.

e. Test across multiple screen sizes

Due to the need for testers to consider testing applications across different platforms and devices, it is equally important to consider the different specifications and sizes of various devices. Screen size is an important component of the testing process as it can display errors that have a negative impact on the application layout. When an application cannot make appropriate adjustments in a specific dimension, it can affect many elements, including some key features missed on the screen, lose important parts of its usability, and deteriorate the user's digital experience and overall brand value.

f. Application size and storage testing

Usually, users will avoid installing applications that take up too much space on their devices. Firstly, developers and testers must focus on optimizing the size of the application as much as possible without affecting its usability. In addition, testers should ensure that the application size does not exceed the specific limit of each update or new version, as excessive memory usage may cause end users to stop updating or uninstalling these applications.

g. Application privacy permission testing

Software developers are unwilling to scare off users with too many application permission requests, as excessive permission requests can cause privacy concerns for users. A few years ago, 215000 applications were removed from the App Store due to requests and collection of large amounts of user data or other privacy violations. Testers need to ensure that the application does not seek any unused permissions while using the application. Similarly, testers should ensure that authorized permissions are also removed after deleting the application. In addition, the correct combination of testing permissions is also important.

h. Application upgrade testing

This is very important for mobile application testers. This process helps ensure that the application does not crash during upgrades, which may be due to the development team providing mismatched version numbers. In addition, preserving historical data after the upgrade is crucial for users to determine their preferences saved in previous versions, in order to avoid trouble when using the application. For example, in food delivery applications, taxi applications, or digital wallets and banking applications, users will save a large amount of data, such as card details, addresses, contact information, etc., and historical data must be appropriately retained for each upgrade.

i. Performance testing

As global users shift towards mobile devices, the performance of each application they use will affect its acceptance and popularity among users. Therefore, performance testing has become a major element in the application testing pipeline, and testing teams should examine application response time and latency to track and predict performance across numerous scenarios and use cases. When a large number of users are using the application simultaneously, it is necessary to check the performance of the application and obtain accurate production forecast values by causing significant load on the server.

4、 Testing equipment

Choosing appropriate tools and setting up appropriate testing environments for automated mobile application testing is crucial. Arrange testing platforms or environments according to the requirements of the tested application. In some scenarios, a test bench can be a combination of a testing environment and testing information. To select a suitable test bench for mobile application testing, any of the following methods or combinations can be used:

a. Simulator

Simulators mainly use hardware or software to make a computer system (called the host) behave similarly to another device or program (called the guest). Simulators generally allow software or external devices designed for client systems to run on the main system. A virtual machine (VM) is a computer system simulator that simulates a complete computer system with complete hardware system functionality and runs in a completely isolated environment through software, providing the functionality of a physical computer.

Common Android emulators include android studio emulator，bluestack，genymotion，Nox App Player，MEmu App Player，koplayer

The commonly used iOS emulators include the xcode simulator

Common virtual machines include Parallels, VMware Fusion, VirtualBox, and QEMU

Generally speaking, there is no need to deliberately define and distinguish virtual technology. Testers focus more on the results brought by this technology, that is, creating an environment that conforms to the software/hardware of the testing equipment through software/tools. There are various configurations of mobile phones and tablets in the market, and virtual devices often cannot replace real phones. They are often used for debugging in the development and testing stages. However, for PCs and Macs, system factors are generally the key to product operation, and development and testing tend to use virtual machines to complete work tasks.

b. Real machine

Real hardware devices equipped with operating systems can be computers, tablets, or mobile phones. Testers often discover compatibility issues in real-world scenarios when testing real machines from the perspective of end-users. Purchasing from the market or leasing from equipment platforms is a common method of acquisition. Real phones are commonly used for testing mobile devices such as smartphones and tablets.

c. Cloud testing platform

Cloud testing is a new model that provides testing services based on cloud platforms. Targeting enterprises and developers, we provide cloud deployment and use of testing tools, equipment, and engineers to meet the full cycle testing needs of enterprise software and systems, including functionality, compatibility, performance, and security. Through network protocols and other means, it is possible to connect and host the target real machine to the data center locally for testing activities. It has the characteristics of multiple devices, convenience, low maintenance cost, and ready to use model. In general, cloud testing platforms provide real mobile phones and tablets hosted in the data center.

strategy

After we have the above testing reserves, we will customize the testing strategy for the project based on project requirements, architecture implementation, project delivery time, project cost, and other factors.

1. For the implementation of project architecture, choose a testing plan, such as mixed development of mobile applications, combining automation and manual use, and combining simulators with real machines for functional testing.

2. Select testing methods based on project delivery time and requirements. If developing a brand new mini program for three months, testing methods are required: functionality, compatibility, performance, security testing, and user experience testing.

3. Select testing devices based on the market ranking of users using mobile application devices, or based on their initial market launch or phased iteration of new features.

summary

Sharp tools make good work. As a software tester, it is recommended to first think clearly about the testing strategy for this task before receiving it, determine the overall direction, practice and continuously improve accordingly, and the quality of the delivered project will be guaranteed to a certain extent from the beginning.