What is iOS App Testing?
iOS app testing is the process by which an iOS application is tested on Apple devices like iPhone and iPad. The goal of this process is to evaluate whether the app works as expected for specific user actions and under different hardware and software configurations. The actions may include installation time, load time, user interface, user experience, appearance, behavior, functionality, and performance.
Why Test iOS Apps?
Apple iOS is one of the most popular platforms for smartphones, tablets, and other mobile devices, with a market share of around 30 percent worldwide, and more than 50 percent in North America. This means that delivering apps that run flawlessly on iOS is critical for ensuring a positive experience among your many customers who use iOS devices.
iOS app testing addresses this need by allowing developers and QA teams to evaluate how iOS apps behave based on different device types and configurations. You can test native, web, and hybrid apps, and you can evaluate their behavior and performance across a variety of browsers, iOS versions, iPhone and iPad device types, and more.
iOS Testing for Native Apps
Native apps are apps traditionally built-in Swift or Objective-C using the native APIs provided by the official iOS SDK (software development kit). They leverage the native APIs and UI components that Apple provides in the SDK.
This means that, in general, native iOS apps work in a relatively consistent way. Still, there could be compatibility issues between different programming language versions, operating system types, and so on. IOS native app testing alerts developers to these problems before the apps are deployed into production.
Web App Testing for iOS
Web apps are mobile versions of websites. As such, web apps can be designed with a variety of frameworks and their behavior can vary widely between different browsers. Testing web apps ensures that your iOS users have a smooth experience regardless of their device configuration.
Testing Hybrid Apps on iOS
Hybrid apps combine the features of native iOS apps and web-based apps. Typically, hybrid iOS apps include a native UI component known as a “webview,” which can be embedded into native apps. A webview gives users a transparent window into Web content from any URL, even local URLs.
Hybrid apps allow developers to build some or all of their app logic using Web technologies, but retain access to some of the native APIs like camera or media library access.
The complex nature of hybrid apps means that it’s especially important to test them for compatibility or performance flaws before putting them in the hands of users.
Types of iOS Application Testing: Manual vs. Automated
You can perform iOS tests either manually or using test automation tools.
Manual testing for iOS apps
Manual testing means testing “by hand.” It involves running the app on a physical device or in an iOS simulator, then navigating through different predetermined test scenarios.
While this approach to iOS testing takes a lot of time, it can be useful in certain situations. For example, manual testing with actual consumers may reveal user interactions that test engineers would otherwise not anticipate.
Manual testing can also be valuable early in the development process if developers need a quick, “one-off” look at a certain facet of application functionality. Rather than having to write an automated test script, they can just open up the app and run the test manually.
That said, because manual testing is very difficult to scale, it’s not a practical way to test applications systematically. Instead, manual testing should be reserved for specific, limited testing needs.
Automated testing for iOS apps
Automated testing is a much better way to test iOS applications systematically. With automated testing, developers and QA engineers can run a suite of tests to evaluate the application every time a change is made to its code. This means they can verify that the application continues to behave and perform as expected and that any problems can be detected before the application is deployed into a production environment.
There are two main ways to perform automated tests:
Unit tests: Unit tests evaluate individual components of an application in isolation. The test typically passes a series of variables to the component and checks that the component returns the correct values. Unit tests help to fix bugs early in the development cycle, and they save the cost of having to correct problems later in the design cycle. OCUnit is a unit testing framework for Objective C in macOS. The biggest advantages of an OCUnit framework come from tight integration into the XCode development environment.
End-to-end tests: In an end-to-end test, the entire application stack is tested, from the UI to the app logic to network requests and backend server functionality. The goal of end-to-end testing is to evaluate how the application as a whole would behave when placed in the hands of actual end-users.
Typically, development and QA teams will run both unit tests and end-to-end tests. They perform automated unit tests earlier in the development lifecycle to identify problems triggered by specific units of code. Then, they run automated end-to-end tests before deploying a new version of the application to detect issues that unit testing may have missed.
Test Automation Frameworks for iOS Apps
There are several tools available to help write and execute automated tests for iOS native, web, and hybrid apps:
Appium is an open source test automation framework that can run automated iOS tests based on XCUITest. The advantage of using Appium instead of testing via XCUITest directly is that Appium also supports Android and Windows tests – so, with Appium, you can use a single framework and set of tests to test apps across multiple platforms.
XCTest and XCUITest is a testing library introduced by Apple in 2015. It’s designed primarily for testing UI elements. XCUITest is the closest thing to an “official” iOS test automation framework.
Earl Grey is Google’s answer to XCUITest for testing iOS apps, and is an iOS UI Automation framework which also happens to be open source! (However, using Earl Grey does report anonymous statistics back to Google). Earl Grey is for iOS only, and tests must be written in Objective-C or Swift. The main benefit of Earl Grey is that it brings Espresso’s synchronization features to iOS testing, to once again ensure that the automation is not trying to do something in the app while the app is busy.
Detox is a JavaScript-based test framework for React Native Applications only which, like Appium, aims to be cross-platform. It is meant to be used as a Node.js library in your test code, so tests must be written in JavaScript. Detox is also fully open source.
Some of the iOS testing frameworks, including Appium, enable tests to be integrated with a CI system. That way, they can run automatically in the context of agile development processes.
iOS Beta Testing
You can run manual and automated iOS tests with internal users. However, if you want to see how real-world external users will interact with your app, you can do so using iOS beta testing. IOS beta testing allows you to invite third-party users to run your app before it is released to the public. Apple’s TestFlight program makes it easy to recruit beta iOS testers.
Beta testing is not a substitute for running your own unit and end-to-end tests. You should run those tests before opening up your app to external beta users. Then, performing beta tests prior to app release can help you collect data about application functionality based on usage by real-world users – who, unlike internal testers, have no upfront knowledge of how your app works or how to use it. For that reason, external beta testers may reveal issues that your own, internal testers overlooked because they did not use the app in the same way.
How to Create an iOS Testing Strategy
To plan an iOS testing strategy, start by asking yourself these questions:
Which devices do you need to test for? If your app only runs on smartphones and not tablets, for example, then you know you can test just for iPhones and not iPads.
Do you need real-device testing? While many iOS tests can run in simulators, tests that evaluate specific hardware features (like networking performance or biometric input functionality) are best run on physical devices.
Which tests will you automate? In general, the more tests you automate, the better. But given limited engineering resources, you may need to pick and choose which tests to automate. Prioritize tests that you need to run most frequently, as well as those that are most critical for guaranteeing a positive end-user experience.
Which iOS test automation framework will you use? As a widely-used open source testing framework, Appium is probably the best choice if you need a cross-platform test framework. If you develop only for iOS, you can run tests directly via XCUITest.
Who will write your iOS tests? If you have a QA team, they’ll likely take the lead in iOS testing. But you may want developers to lend a hand in helping to write tests, too.
When you know the answers to these questions, you can plan your step-by-step approach to implementing iOS tests.
iOS Testing Best Practices
Consider the following tips to make your iOS tests as effective and efficient as possible:
Run tests in parallel: Instead of running tests one-by-one, use a test grid to run many tests at once. You’ll get results much faster, which means you can get your app to users more quickly.
Test on real devices: Although testing on simulators makes sense earlier in the development cycle, testing on real devices leads to more reliable test results. If you don’t have real Apple devices on hand, you can access them using the Sauce Labs test cloud.
Consolidate your tests: If you need to perform cross-platform tests, it makes sense to use a framework like Appium so that you can write tests once and run them across multiple OS platforms.
Don’t neglect environment configuration tests: Because iOS environments tend to be more consistent than Android environments, it may be tempting to assume that all iOS users will run the same environments, and that hardware is the only big variable in app performance. But that’s not true. IOS users can install alternative browsers, modify OS settings, and even “jailbreak” their devices. Ideally, you’ll be able to test for all these variables to ensure that your app runs flawlessly no matter how your users have configured their devices.
Run unit and end-to-end tests: As noted above, unit tests and end-to-end tests shouldn’t be an either/or proposition. Run both to maximize your ability to detect problems before they impact your users – and to ensure that you can identify bugs as early in the development process as possible, when they are easier to fix.
Getting Started with iOS Testing
Get started running iOS tests quickly and easily with Sauce Labs, which provides access to a complete selection of simulated and physical iOS devices and configurations. Sign up for a Sauce Labs free trial to see how it works.
