In July 2013, an article from Gizmodo reported the world-market to have about 12,000 types of Android powered mobile devices. It further informed that these devices were running on eight different forked versions of same OS.
Three years have passed since then, two new Android versions came out, had numerous updates, shells and so on. In addition to that, several new companies have appeared and actively developed, getting their market share big enough to consider them major players.
What does this mean to Android app testing? It just gets more complicated or more interesting, depending on what you like more.
1. Get most popular physical devices
Although for android testing - the perfect scenario is to get as many devices as possible, on practice its is so expensive that only largest companies can afford the luxury. So the most optimal way here is to get the most popular devices on the market and use them for testing. LG, Samsung, Google, HTC and Sony - take one of each and you are good to go. They are stable, they last long and their recent devices will surely support upcoming updates.
2. Always test on various Android versions
Android OS versions list is almost as large as device market. There are still devices using Android 4.x, lots of devices using Android 5.x, even more using 6.x and Android 7.x came out not so long ago. To get wider the range of customers happy consider supporting and of course testing on as much OS versions as possible.
Also always remember that if you have a start-up at the beginning of its way and you are limited in funds, your friends and family is also a good polygon for testing! You can always borrow their Android device and test your app on it a bit, maybe ever run a set of smoke tests. From our experience, even a small period of time spent on it will provide you plenty of useful information starting from “Does it work on Android 4.x at all?” all they way to speed, UX etc.
3. Do not use emulators for manual testing
What is suitable for development is not always good for QA. And there are lots of reasons for that: you can emulate how it looks, but not how it feels - using it with full size keyboard and mouse will never get you a real life experience. You will never get a call or an SMS on the emulator, it will never run out of battery or loose connection.
Same with software, testing an Android app on a device will get your app affected by various sets of 3rd party hardware, like anti-virus apps, apps that manage devices memory and storage etc. which you'll not get on the emulator.
4. Use automation
Depending of the app itself, your release frequency, development process and financial aspect you may want to start using automation for testing your app. There are lots of pros in it and not as much cons as may seem from the first glance.
Overall using test automation is the best way to reach an optimal level of test effectiveness, test efficiency and testing coverage. The ultimate value of test automation comes from avoid manually conducting repeatable test runs and free up time for new features and do faster releases for your customers.
Android app automation testing will increase your test efficiency as you will get more test runs in certain amount of time and shorten your time to market especially if you are using short sprints and your business is customer-driven - the faster you apps passes all needed tests before release the faster you will get you will satisfy your customer's needs.
Here is a short video about introducing to appium
5. A few aspects that make mobile app testing differently from the desktop testing
A desktop application is tested by using the central processing. On mobile devices, the applications are tested on cell phones like LG, HTC, Apple and Samsung.
Mobile device screen size is smaller in size than desktop, but is often greater in display resolution.
Mobile devices have less RAM and ROM than desktop.
Mobiles use network connections like GSM, WCDMA, LTE, CDMA, WiMax and WIFI where in desktop usually is used LAN connection.
There many differences between using automation testing on PC and mobile devices on various platforms.
6. Some kinds of test which are mostly required for executing testing
There are a few required cases for testing mobile devices. Battery usage - the important parameter for mobile device as usability (it means also autonomy) and it is very important that the application uses as less as possible system resources that will give increased battery usage time on a single charge. Response speed of the application - the time on different devices, less is better RОМ and RAM - to install the application and its normal functioning. Functionality of the application – the application is corresponded to the requirements, properly responds to external interrupts, no crash because of: the network losses, the transition to landscape mode, power down etc.