Getting Started with Automated UI Testing – Picking the Tools

Starting the journey to automated testing is difficult to navigate. There are so many choices for different automation tools, assertion libraries, testing frameworks, and even choices of what to test, it can be overwhelming to try and choose the right place to start. This article will address different user interface (UI) testing frameworks and the technology that you can choose from, and help you to understand the different components that go into a complete framework.

Perhaps you are a small development shop that is responsible for both the development and testing of your app. Maybe you are in the position in your organization where you are asked to make a decision about how you can automate the testing of your software, though you’ve never had to do this before. You may even be a manual QA tester that is looking to start automating application testing, but with little or no experience, you need direction in terms of which software to use, and where to start your testing journey. 

Like many aspects of the software world, the options are plentiful and varied, standards and tools are constantly evolving, and it’s hard to know where to start. Do you want a framework that includes Selenium or Appium automation tools, or should you use a JavaScript testing framework like Cypress or Playwright? What types of tools, libraries, and plugins do you need to consider?

Different testing tools cover different types of test environments. If you only need to test desktop web applications built with the latest technology stacks, like Angular, ReactJS or Vue.js, you might have heard more about frameworks like Cypress, Playwright, Puppeteer or Test Cafe. 

These frameworks are more developer-centric, give more access to the source code of your web application and interact with the browser in a different way then Selenium-based frameworks do. It's important to research if the framework you are using supports all the requirements you have to test your application. Sauce Labs provides support for these types of developer-centric JavaScript frameworks with the saucectl CLI.

You can also choose to use Selenium or WebDriver automation tools for browser automation tests, which have the ability to communicate and automate a nearly every version of every browser, paired with a programming language such as Python, Ruby, JavaScript, C#, or Java. With these options you can use Selenium commands, or use a framework such as Selenide (with Java) or Watir (for Ruby). In each programming language, you have a different set of choices for test runners, which are libraries that help you check assertions, as well as orchestrate your tests, managing when certain logic is carried out and when different tests are run. As an example, if you write your test code in Java using Selenium commands, you can use the JUnit4, JUnit5, or TestNG test runners. If you are writing your tests code in Python with Selenium, you can use PyTest.

Some testing libraries also provide support for different types of testing besides User Interface (UI) testing, so you will want to consider what your testing strategy will be before choosing all the pieces of your framework.

If you have an application that needs to be tested on not only desktop browsers, including older browsers, as well as on mobile browsers and on mobile apps, and you need more than some JavaScript testing frameworks can provide, WebDriver based tools like Selenium- and Appium-based frameworks may be a better choice. With these types of frameworks, you have  the option to take desktop browser tests and modify them for mobile devices. 

Appium can be used for both iOS app testing and Android app testing, and also pairs with programming languages such as Java, Python, Ruby or JavaScript, using a test runner or assertion library. In many cases, it may be relatively simple to start with Selenium tests for a web app, then modify the code with Appium commands, adding on tests for functionality that are specific to mobile devices. Keep in mind, however, that Web, Android, and iOS applications may have very different functionality, meaning that you will have to do some overhauls of your tests for each environment.

There are also the Espresso (Android) and XCUITest (iOS) families of testing libraries, which can be included as a part of an application’s code package. Android applications written in Java or Kotlin can use Espresso commands to test your application, while iOS developers can use XCUItests alongside their application.

There are different advantages and disadvantages to the different sets of tools and frameworks, and it’s important to work with an experienced QA Engineer or developer before you choose an overall testing plan. Keep in mind, that like all software, your needs and use cases will change over time, and that you need to be flexible with trying new and different solutions for your quality testing.

Choosing a well-supported technology such as Selenium or Appium can be helpful as there is a wide community of support, as well as more frequent features and updates; however, there can be a trade-off in terms of ease of use. With more choices comes more responsibility, and Sauce Labs offers support for a wide range of options to help you have confidence that you are delivering the best possible experience to your customers. 

It’s impossible to recommend one framework overall as each company or application’s testing needs are different, as are the engineering knowledge and expertise available on your  teams. Start your automated testing journey by learning test automation, see how you can run your existing tests on Sauce Labs.