Best Practices for Shifting Accessibility Testing Left

QA professionals, testers, and developers are constantly learning new tools, tech stacks, and development practices. When they’re told they have to learn accessibility, it can often feel like an unwelcome and overwhelming disruption, slowing them down and forcing them to test and rewrite what they thought was perfectly good code.

The good news is accessibility tools are more tester-friendly than ever. Automated accessibility tools fit seamlessly into modern development environments, and are a great way to shift accessibility left in the Software Development Lifecycle (SDLC), bringing accessibility testing closer and closer towards design. 

In this blog, I’ll share four best practices for shifting accessibility testing left. I’ll also cover how to integrate axe, Deque’s open source accessibility tool, in NodeJs Jest with the Selenium browser automation tool and Jest.

Traditional accessibility testing is often thought of as slow and manual and done at the end of the SDLC in user testing, after production. While manual accessibility is still a necessary part of testing, 50% of accessibility defects can be caught with automation. 

Nowadays, there are accessibility tools that are built to integrate into every part of the software development lifecycle, from development to testing to when your site or application is released. These tools are built by and for dev teams with modern development environments, so they won’t slow down your development velocity.

Illustration of where tools fit into each role of the software development lifecycle: accessible design for designers, axe DevTools for front-end developers, test automation engineers, senior developers and product owners, axe Auditor for QA testers and accessibility testers, and axe Monitor for central accessibility leaders.

Shifting accessibility left in the SDLC can be done easier and faster with automated accessibility tools, and it also saves development time and resources. IBM studies have shown that defects caught post-production are 30x more expensive than in design, coding, and testing. Small investments of time and resources in development for accessibility will ultimately save your organization money and ensure less disruption to the SDLC.

Diagram showing small investments for accessibility during backlog, design, coding and testing (i.e. shifting accessibility left) vs. large investments for accessibility during release (traditional way accessibility is practiced).

If developers and testers can automate something, they will. When accessibility can be automated, developers and testers are much more likely to start incorporating accessibility into their development process. 

Until recently, “automated accessibility testing” meant cumbersome “scan and report” tools that could only be used once your application was practically ready to be deployed (if it hadn’t been deployed already). Dev and testing teams would get long lists of accessibility issues that could require a fundamental reworking of components, causing expensive and frustrating slow-downs in the development cycle.

Luckily, accessibility tools have come a long way, and developers and testers can now access automated accessibility testing tools that can run as part of their existing workflow. Issues are raised while they’re coding, making fixes much more efficient. With free tools like axe, you can start testing for accessibility in minutes.

Axe-core is Deque’s open source accessibility rules engine, which powers a number of accessibility tools from Deque. Perhaps most notably, axe-core powers the axe browser extension available for Chrome, FireFox, and Microsoft Edge.

It’s important to note, however, that automated accessibility testing tools can only catch 50% of your accessibility defects. Your application will still need to undergo manual testing with assistive technology tools, especially if your organization has any concerns about compliance with accessibility regulations or legal risk. 

It used to be that manual testing could only be done by an accessibility expert who is familiar with assistive technology. Now, the axe suite of tools offers Intelligent Guided Testing (IGT), to help guide developers, testers, and product owners through the testing for accessibility issues using a wizard based approach that removes the necessity to be an expert in accessibility. IGT will improve your coverage to 80% without needing specialized knowledge or expertise.

To recap, using the automation in the axe browser extension is a great way to start testing for accessibility. However, it’s important to note that manual testing or IGT is needed to move towards full accessibility.

Some manual accessibility testing can’t be done with automated tools. For example, using a screen reader or doing keyboard only testing using the tab key to simulate the experience for users with disabilities on your site or application.

Although some things around accessibility must be validated manually, such as alt text, you can still write simple unit or integration tests to validate the alt text so that future releases do not break the accessibility without being caught. Furthermore, if you have the design information, it’s easy for testers to proactively include accessibility requirements into automated tests.

For example, the code below is testing the Deque logo and the alt attribute associated with it:

1 <// Check the Deque Logo

2 ok(img.src.indexOf('logo.png' !== -1, 'image is the Deque logo image'));

3 equal(img.getAttribute('alt'), 'Deque Logo');>

Testing is ultimately about risk mitigation, so you need to figure out how often you want to retest things manually to verify that compatibility has not changed. Usually, this is only necessary when there is a major update released for screen readers, operating systems, or browsers.

An important part of getting started with accessibility is having the right tool. The second important key to accessibility is training. The sad truth is that testers and developers don’t learn about accessibility in professional training. Additionally, most developers or testers have never witnessed a person with a disability using an assistive technology, and having a firm understanding of accessibility is key to long term success with accessibility. 

Outside training creates the fundamental foundation that is critical for building a culture of accessibility. Furthermore, beyond automation and accessibility, there are nuances to accessibility that are easier to learn from experts in hands-on training. For example, there are many intricacies related to single-page applications, how to use screen readers for testing, building tests and scripts for accessibility, building a constructive and useful accessibility issue log, etc. 

Now that we’ve discussed the best, easy steps to shift accessibility left in the SDLC from production to development and testing, we’ll review how to use axe-core to run web accessibility tests in NodeJS based projects with the Selenium browser automation tool and Jest. 

This example below assumes you have Node Js and Npm installed.

The first step is to install the proper package into your project, by running

npm i axe-webdriverjs

Once installed you should see the following in your package.json:

 “dependencies”: {

    “axe-webdriverjs”: “^2.3.0”,

    "selenium-webdriver": "^4.0.0-alpha.7"

  }

Now your dependencies are set up, and you need to pull in the right classes and set things up.

In any file that runs a UI test using Webdriver, import the classes required:

var WebDriver = require('selenium-webdriver');

var chrome = require('selenium-webdriver/chrome');

var AxeBuilder = require('axe-webdriverjs');

And set up your driver with specific options you want like so: 

Var driver;

beforeAll(async () => {

driver = new WebDriver.Builder().forBrowser('chrome').setChromeOptions(options).build();

});

Note: we have told the library to use the standard WCAG 2.0 A and AA rule sets which will report on all violations excluding best practices, experimental rules, etc.

Now, wherever your tests set up the Webdriver instance and/or cause UI states to change, make calls to the library to analyze the page and generate reports:

test('should test for what the overlay does (add specific test scenario)', async () => {

await driver.get('https://dequeuniversity.com/demo/mars/').then(function() {

    AxeBuilder(driver).analyze(function(err, results) {

      if (err) {

        // Handle error somehow

      }

      console.log(results);

      await expect(results.violations.length).toBe(0);

    });

  });

 });

There are many more features in the library that allow for customization of scans that includes turning on and off rules, excluding content to scan, and running specific rulesets only. Getting started with this integration will make accessibility a core part of your CI/CD pipeline.

The power of axe is a great place to get started if you’re completely new to accessibility. If your organizations need more coverage and features to help get your team up and running, axe DevTools is the right solution.

Beyond the free axe browser extension, axe DevTools is the enterprise solution that provides:

Starting with free automated tooling in the axe browser extension, in unit tests, or in your CI/CD pipeline is a great way to catch 50% of accessibility issues. Training is also key in building a culture of accessibility to properly build it into your SDLC. If you want to learn more about accessibility, and how to embed it throughout your organization, check out the resources below.

Sauce Labs is also hosting an upcoming webinar on Using Axe to Add Accessibility Checks to Your Existing Selenium tests. Join the session and learn how to put this into practice.

There are many ways to get started with accessibility testing in the software development process, such as using the axe browser extension, building unit tests for manual accessibility reminders, and investing in training. To take it one step further, you can use the axe and Selenium integration outlined above in your CI/CD pipeline. 

Using free, open source tools to practice automated accessibility testing early in your SDLC will help shift accessibility left — ultimately saving your development team time and resources while making the web a more inclusive place for all.

Dylan Barrell is the Chief Product Officer at Deque. He has almost 20 years of experience in technology development, technical sales, product management, product marketing, and corporate development. Dylan founded a startup in 1989 that was acquired. He then went public with OpenText and helped grow revenues to over $400 million. He later led the effort to create a digital center within a Borders store and to begin the transition from purely physical products to digital and physical product mix delivered in a cross channel online and in-store way. In addition to his work experience, Dylan has an MBA from the University of Michigan and a BS from the University of the Witwatersrand. Dylan believes in leading through action to create value for shareholders and the community.