If you're testing apps for iOS 9.3 and above, there's a good chance that you are or will be using the XCUITest framework. In this post, we'll take a look at some XCUITest best practices, and how to incorporate them into your test regime.
XCUITest
Let's start with a quick overview of XCUITest:
XCUITest is part of the XCTest testing framework, which in turn is part of Apple's XCode development system. XCUITest allows you to automate testing of the user interface and overall user experience. Note that as of XCode 8, the older UI Automation test framework will no longer work. You will need to refactor any UI Automation tests to run under XCUITest.
UI Testing: Why It's Important
UI testing is (or should be) a major part of the test regime for any user-oriented application. While it resembles code/unit testing in overall outline and approach, it differs from functional/code-oriented testing in some fairly specific ways. The goal of UI testing is not to look at the application's internal operation. It is, rather, to look at the application from the outside, and to test the application's response to user actions.
In this way, it is similar to API testing—UI testing and API testing both focus on interface-based interactions with your app. In the case of UI testing, however, the interactions are not code-based API calls, but user actions by means of iOS device inputs.
Basic UI Test Patterns
Most functional UI tests follow a basic pattern:
Do something to the app by means of the UI. For an iOS-based app, this would typically involve tapping, pressing, swiping, or other screen-based inputs.
Record or capture the app's response. This can include both screen recording and capturing the state of internal application data.
Measure the response against an expected response (by means of an assertion), then record or report the result.
A full UI test would typically consist of a sequence of UI-based actions, with each following the above pattern.
Performance-based UI tests follow a somewhat different pattern:
Do something via the UI - an action or sequence of actions.
Record the time required to complete the action and expected response.
Repeat the action or sequence a set number of times.
Report the average completion time, plus outliers and other relevant statistics.
Best Practices
XCUITest includes the basic features needed to perform UI tests following the patterns described above. As is the case with all test suites, however, the value of the tests depends strongly on test design, implementation, and analysis—all of which are ultimately the responsibility of the testing team. Here are some best practices for UI testing in the context of the XCUITest framework:
UI Recording is Your FriendUI recording is one of the most important built-in features of XCUITest. It allows you to record a test interaction with your app's UI, and save that interaction as test source code. You can then edit/tweak the code to add test details (including XCTest assertions), much like recording a macro for later editing and expansion. It may not be macho programming, but it saves time and trouble, and provides you with a reliable framework of generated code.
The UI test recorder is also useful as an ongoing test tool. It provides information about interactions with the UI, including application internals.
Sequence-Dependent or Sequence-IndependentRecognize that UI tests are generally sequence-dependent (a given test step may only be able to proceed if the previous step is successful), and write tests with this in mind. In your test code, self.continueAfterFailure will typically be set to NO to reflect this fact. If you are creating UI tests that are not sequence-dependent (for example, testing radio-button options which produce an immediate GUI response), you can set self.continueAfterFailure to YES, and make the appropriate provisions in the test code.
Test the Full Range of InteractionsXCUITest allows you to set user interactions with considerable precision in most cases. You can, for example, specify tap(), doubleTap(), and twoFingerTap(); you can also set the number of taps. You can specify press() with a duration, and with both a duration and a dragging target.
These interactions include four directional swipes (left, right, up, and down), pinch(), with scale and velocity, and rotate(), with velocity. Different users have different UI interaction styles; you can make full use of these features to develop a more detailed picture of the results of such varied interactions.
Don't Forget AccessibilityAlong with standard UI testing, test handicapped/hearing-/visually impaired accessibility using the UIAccessibility protocol. In part, this is a matter of good sense and proactive customer service, since it maximizes your potential customer base.
It may also turn out to be a practical necessity. Compliance with accessibility standards is becoming an increasingly common requirement for entry into many significant markets (including software for government use).
Go Beyond Standard Functional and Performance TestingAlong with testing for errors and failed responses, test for unexpected or unwanted results which may not show up as errors. These could include accidental activation of unrelated features, actions that result in items being hidden or partially obscured, or other responses that interfere with the user experience. Remember: if an app works, but is too annoying to use, most people won't use it.
XCUITest: Making UI Testing Easier
It is no secret that UI testing is not the favorite pastime of most developers and test teams. It is not always easily automated, and the results are not always easy to collect and analyze. It also includes an unavoidable element of subjectivity at the user-experience level, and even relatively objective functional UI tests may fail to fully capture the reasonable range of user actions.
The XCUITest framework includes features which make it easier to automate UI testing, as well as precisely specify user actions. While best testing practices are ultimately up to test designers and testing teams, XCUITest is of considerable value in implementing such practices.
Michael Churchman is a contributor for Fixate IO. Michael started as a scriptwriter, editor, and producer during the anything-goes early years of the game industry. He spent much of the ‘90s in the high-pressure bundled software industry, where the move from waterfall to faster release was well under way, and near-continuous release cycles and automated deployment were already de facto standards. During that time he developed a semi-automated system for managing localization in over fifteen languages. For the past ten years, he has been involved in the analysis of software development processes and related engineering management issues. He is a regular Fixate.io contributor.
