10 minutes to read

Top Mobile Automation Testing Tools: Open Source, Android, iOS

March 26, 2019

Developing mobile applications is very different from developing desktop or even web software. But automating every kind of mobile application is neither possible, nor required. It should depend on what kind of application do you have, if it has complex logic does it have, and how long it will be in development or getting big updates. If you decide to have mobile automation, you will face another huge pane - ‘What tool should I use for this purposes?' There are lots of mobile test automation tools are available to fit mobile device test automation process. But let's take a look on the most popular (used) of them and help you to decide.

Top Mobile Automation Testing Tools

#1 Appium

Appium - open-source test automation framework for mobile native, web and hybrid apps. Appium is best option for QA teams that are trying to test the functionality of native, mobile web and mobile hybrid apps across iOS and Android because of it uses the WebDriver JSON protocol to standardise the automation API across the two platforms and three app types. It uses first-party automation instrumentation and does not require modifications to the source code. Appium let's you choose the language or framework you want to write your test in.This tool is less useful for developers who want to develop and perform unit testing because it uses a different scripting language than the app itself, (e.g. Objective C or Swift). Reports generated by Appium are less informative than a debugging and fast feedback loop perspective, and does not include videos, network logs and key vitals information.

#2 Calabash

Calabash - designed for behavior-driven development (BDD) workflows test framework based on Ruby development language. The tool offers an easy way to develop the features in parallel with covering them by the auto tests in an easy user-flow based language. Calabash is appealing for both developing and test automation for mobile devices purposes. It provides comprehensive insights and reports to Dev and QA teams. This tool can be used with Cucumber but it's not limited to Ruby. You also can use Java, C#, Flex or other languages. Another great stuff about Calabash, that it was created by the Xamarin team and because of this you can easily integrate you test scenarios to execute on over 1,000 real mobile devices in the cloud.

The common features of Appium and Calabash:

They are free;

Both have a large open source communities;

Supporting run test suites on both simulator and real devices;

Cross platform compatibility.

#3 Selenium

Selenium is a portable software testing framework for testing mobile application

Selenium - the leading open-source test framework for web app test automation. It's one more tool that can be used for mobile app test automation.

The Selenium framework is the best option for web test automation teams testing for responsive web design, or stand-alone web sites. It's less useful for developing unit tests, which makes this framework less appealing for developers. Core Selenium test reports are not very informative and lack unique mobile related insights.

Also Selenium has a project that is directly focused on mobile testing called Selendroid. It is a mobile application test automation framework which drives off the UI of Android native and hybrid applications (apps) and the mobile web. The main advantages of Selendroid are that it using the Selenium 2 client API, that makes scenarios easy writing, can interact with multiple Android devices at the same time via full integration as a node into Selenium Grid for scaling and parallel testing and don't require any changes of application in order to automate it.

#4 Robotium

Robotium - is an extension of Android test framework that allows to create UI tests of native and hybrid Android applications. It can be run on both emulator and real device.

The major benefits of this tool are:

Scenarios are easy to write;

Fast test execution;

You can delay and timing use test runs;

Automatic views finding;

Auto scrolling and other helpful staff;

Code looks nice and short;

You can create scenarios in black-box style (without knowing of code implementation).

Major disadvantages of Robotium are that it suitable only for Android apps and in case of cross platform testing you need to look for other tools or create auto tests for iOS in second tool from scratch; Robotium doesn't work with web or flash; old versions can be really slow.

In conclusion, we can say that if you are developing application for both iOS and Android, better choice for mobile test automation will be Appium or Calabash. Both of them a open sources and can cover most of issues. Appium is easy to start because it does not require any changes in application code. But Calabash is a little bit more stable and has a great integration with Xamarin cloud in case you don't have lots of devices in your office to perform testing. But if you are developing mobile application for some specific platform you are free to choose from plenty of automation tools that are focused on it, about this tools we want to say below. Let's consider some of the best android automation testing tools and most popular ios automation testing tools.

Best iOS automation tools

#1 Xamarin.UITest

The testing tool activates Automated UI Acceptance Tests written in NUnit that are run against iOS applications. It assimilates closely with Xamarin.iOS and Xamarin as well. It might be used with iOS projects. These projects must be written in Objective-C/Swift and Java. The NUnit tests can be accomplished on iOS devices. When user enters a text or tap any button this tool interacts with user interface.

Test class that is called fixture comprises the tests (as UITest is written as a method that is referred to as a test).It can contain a single test or group of tests. The fixture is in charge of any setup to make the test run and any cleanup that needs to be carried out when the test is finished.

The test fixture includes either a single test or logical complex of tests. It is liable for all the setups to make the test run and any cleanup that needs to be performed when the test finishes.

Any test must keep to the Arrange-Act-Assert pattern:

Arrange - Conditions will be set up and things will be initialized. In the wake of it the test can be actioned.

Act - The test will coact with the application, enter text, pushing buttons, and so on.

Assert - The test views the results of the actions accomplished in the Act step to spot the regularity. For instance, when a particular error message is shown the application may verify it. You can load the tests to Xamarin Test Cloud where the tests are run on pre-selected devices. Or you can run the tests locally against a device or simulator by applying the Test Runner in Xamarin Studio.

#2 Frank

The next iOS automation tool is Frank. Test scenarios that are written in English-like statements are used in this text and are maintained by Cucumber framework. Frank needs to be carried inside the source code and its commands are combination of Cucumber statements and JSON commands. It offers large community support, easy to understand English-like statements and expanding libraries.The ‘Cucumber Recipes' book has a nice step-by-step recipe called Test iOS Using Frank that is able to help for new automation testers.

Saying about disadvantages, Frank is not considered as simple tool to use because it is difficult to run on devices and it has limited support for gestures like pinch and zoom etc. Additionally, in order to run Frank on real devices, it is required to make changes in the configuration files which could be a large issue in certain occasions.

#3 iOS-driver

This ios automation testing tool gives you an opportunity to automate any native iOS, hybrid, or mobile web application. It is absolutely compatible with the Selenium and Webdriver API. Therefore the iOS automation is as simple as automation for browser.

As this tool can wholly integrate with Selenium Grid you are able to reuse your web automation infrastructure including your helper and utility classes. It can be data creation or page objects.

As one of the ios test automation tools it has the next features:

It can implement the JSON wire protocol;

You are able to apply emulators for running it;

It can be run as a regular node in a Selenium Grid;

You do not have to change the sources or jailbreak the device;

Additional apps are not compulsory.

But using simulators is the default way to use ios-driver. It offers more pliancy, and it is faster than tests on a real device.

Top Android automation tools

#1 Espresso

Espresso is considered as the recent Android test automation framework that got open-sourced by Google, making it available for developers and testers to hammer out their UIs. Espresso has an API that is small, easy to learn and built on top of the Android instrumentation framework. You can rapidly write concise and trusted Android UI tests with it. It is supported on API level 8, 10, 15 and afterwards. It's quite trusted, synchronizing with the UI thread and quick. But it also with has no support for webviews as Uiautomator.

#2 Robotium

We already have mentioned this tool in our previous post. But it also match criteria of best Android automation tool. Let's take a quick look. It was the most widely used Android testing framework in the early days of Android. A resemblance with Selenium in Android makes testing API quite simple. Robotium is an open source library extending of JUnit with wide range of useful methods for Android UI testing. It provides powerful and robust automatic black-box test cases for Android apps (either native or hybrid) and web testing. Robotium enables you to write function, system and acceptance automation tests.

#3 Selendroid (Android)

Selendroid is also one of the automation tools for android testing which drives off the UI of Android native and hybrid applications (apps) and the mobile web. Selenium 2 client API is used to write tests. It has following features:

It has full compatibility with the JSON Wire Protocol Selenium 3 Ready;

Existing Emulators are started automatically;

You are able to extend it at runtime with your own extensions;

It supports hot plugging of hardware devices;

Selendroid can coact with multiple Android devices simultaneously. It can be emulators or hardware devices;

The concept is the same for automating native or hybrid apps.

Try all of the above mentioned tools and select one you like more to develop quality automation tests. Each of the above tools has advantages and disadvantages so think about your testing need: UI testing, compatibility testing, functional testing, etc. and pick up the best tool for you. Keep in mind that automation test is a hard complex work which require continuous self learning and web research.