There’s no disputing that an automated testing framework is an integral part of the software development cycle. You absolutely need automated testing to deliver quality applications in a timely fashion, particularly for rapidly changing mobile apps.
To reap the greatest benefits from your software delivery, your automated tests should be carefully planned and scheduled in such a way that they are cost-effective. For this purpose, the right choice of tools is critical. It’s also a hard choice to make, because navigating through the vast variety of testing tools available for mobile developers nowadays can be very challenging.
With this article, I’m here to help. Below, I provide a guide for evaluating various Android UI frameworks. The tools are listed in alphabetical order.
Appium
Appium is an open-source mobile testing tool that supports both iOS and Android. It can be used to test any type of mobile app: native, web and hybrid. Appium is a cross-platform tool, meaning you can run the same test on different platforms. To achieve this cross-platform capability, Appium utilizes vendor-provided Android UI framework for testing: XCTest for iOS and UIAutomator or Instrumentation for Android. It wraps these vendor frameworks into Selenium WebDriver. This allows Appium developers to write tests in a wide variety of programming languages: Java, Objective-C, JavaScript, PHP, Ruby, Python, etc. It also makes writing the Appium test very similar to writing Selenium tests.
As WebDriver was initially created for web testing, Appium extended WebDriver with extra API methods to make it more suitable for mobile automation.
Here are some additional useful Appium features:
You don’t need to install anything on the device.
You don’t need to recompile or change your app in any way to test it with Appium.
Appium has a very big and active community.
Appium comes with a tool to scan and analyze an application’s UI components— Appium UI Inspector. Developers can also use Android Studio’s UI Automator Viewer.
If you need to write tests for both iOS and Android, and you’re a Selenium fan, Appium would be a great choice for you.
Appium is supported by Sauce Labs for testing with emulators, simulators and real devices. Sauce is also highly active in the development of Appium itself, with a number of Sauce developers contributing to the platform.
Calabash
Calabash is an open-source mobile testing tool that was developed and maintained by Xamarin. It supports native and hybrid apps on both iOS and Android. The set of actions that can be performed by Calabash testing include gestures, assertions, and screenshots. Calabash is integrated with Cucumber, a testing tool for Behavior-Driven Development.
Xamarin announced in 2017 that it was ending active development of Calabash. While you can still use Calabash for now, the decision to cease development makes Calabash a less-than- ideal testing solution.
Espresso
Espresso is an Android UI Framework from Google. It’s a lightweight, white-box tool targeted towards app developers, which means to use it to its full potential, the test developer has to be fully familiar with and have access to app code. As a result, Espresso tests are very reliable and they run fast. And as Espresso has access to UI objects’ internals, it can test WebView, i.e. hybrid applications. Tests are written in Java. (Duh!)
On the downside, Espresso can test only one application at a time, and doesn’t have access to device resources. However, this could be easily solved by creating combined tests with UI Automator. This blog post explains how to do just that: http://qathread.blogspot.com/2015/05/espresso-uiautomator-perfect-tandem.html
Sauce Labs also supports Espresso for testing on real devices.
UI Automator
UI Automator is an Android UI framework for mobile testing developed and maintained by Google. Its key features include cross-app functional testing, the ability to test multiple apps, and switch between installed and system apps.
UI Automator is a black-box testing tool (i.e., a test developer doesn’t need to know the internal app structure and can rely entirely on visible UI elements). Not surprisingly, UI Automator’s tests are written in Java, Google’s ultimate language of choice. UI Automator consists of two sets of APIs: UI Automator APIs, to manipulate the UI components of an app, and device state APIs, to access and perform operations on the device (change the device rotation, press the back, home or menu button, etc). It also comes with a very useful UI Automator Viewer, a GUI tool to scan and analyze the UI components currently displayed on the device.
The downside of the UI Automator is that it doesn’t support WebView, upon which hybrid Android apps are built. Therefore, UI Automator supports only native Android apps.
Robotium
Robotium is an open source Android UI framework. It has been around since 2010, and by now is a very mature and stable tool. The last version, 5.6.3, was released in Sept. 2016. In the last few versions, test readability and test execution speed have been significantly improved.
Most technical blogs, tutorials, and courses refer to the type of testing utilized by Robotium as black-box testing. I would argue that it is really gray-box testing, because to write Robotium tests, you need to know the internal application structure, at least partially (activities’ names, for example).
Robotium is an extension of the Android testing framework and, as such, it utilizes run-time binding to UI components, which makes tests more robust. Robotium comes as a jar file that has to be compiled with your project.
This is a partial list of the main Robotium features:
Supports both native and hybrid applications
Can run tests on both real devices and emulators
Supports full Android UI paraphernalia: activities, buttons, menus, toasts, dialogs, etc.
Supports gestures
Has some device control functions: change device orientation, take screenshots, unlock screen, etc.
Robotium recorder (https://robotium.com/) is available as a paid Android Studio and Eclipse plugin. It’s a great tool to get up and running with tests quickly.
Can run as part of continuous integration
Robotium’s language of choice is Java.
Robotium is also supported by Sauce Labs for testing on real devices.
Conclusion
Choosing the right Android UI framework for mobile testing for your project is challenging. In some cases, you may need to use multiple tools to strike the right balance. This is why a testing grid like Sauce Labs is helpful as it supports multiple testing frameworks.
Nadia Minaev is a Fixate IO Contributor and a software engineer with deep knowledge in a wide variety of modern technologies. Throughout her career she has worked for such well-known technology leaders as HP, Intel and others. Nadia founded Alkon Technologies, a software development company, and FarmGalaxy, a mobile app that connects local farmers to their customers.
