Continuous Testing in the Retail Industry

In today's retail marketplace, digital channels hold a prominent and often central position. Modern software practices emphasize the need for rapid development and deployment. The retail world is no exception to this need for speed. To match the pace of innovation, continuous testing throughout the software development life cycle is the best way to build digital confidence and provide a great user experience.

What is digital confidence? It means you know that you are creating and delivering the best possible experience to your customers, that your web and mobile applications look and function exactly as intended before your users ever interact with them, and that your customers know they will have a flawless experience every single time they use them.

Retail companies understand that the only way to compete in the market is to offer a flawless digital experience to their customers. With online purchasing increasingly becoming the norm, having a thoughtful testing strategy and process is critical to deliver those great user experiences in order to compete in the market. 

For example, McKinsey & Company found in their 2017 customer experience study that 25% of customers will abandon a brand after just one bad digital experience. These kinds of numbers showcase the direct financial impact of poor quality. Furthermore, a Forrester report published in January 2020 also found that the revenue impact of a 1-point improvement in an industry’s CX index score led to a direct increase in revenue resulting from a yearly incremental increase per customer. This shows us that achieving digital confidence is an investment in your company’s financial success as well. 

To create digital confidence, continuous testing is a good place to begin. However, when trying to achieve a continuous testing practice in a retail company, it can be difficult to negotiate an adoption of behaviors that are more typical to software companies.

There are several reasons for continuous testing:

Let’s explore these in order to understand where continuous testing fits within the retail industry.

One of the big differences between software and more traditional products is that software is constantly changing. Traditional products have high upfront costs to ensure their quality. All of the testing happens before the product is released. You can't change the design of the shoe that is already in the customer's closet. With the majority of software products, we are not only able but obligated to continue to evolve the product after its release. One of the great business advantages of this is your team can very quickly develop a minimum viable product that sales and marketing can get in front of the public for feedback. You can and do make major changes after release, whether it's a redesigned UI, an improvement to the machine learning algorithm, or vast improvements in the ability to scale, all in the ongoing effort to provide a flawless user experience..

The demands of retail require this ability to change as much as any industry. The nature of the industry is to be time-sensitive. There are holidays, sales, and new product lines that can't wait for long development cycles. Think about the speed with which a brick and mortar store changes its atmosphere to reflect marketing needs, whether it's a new window display, a different banner hanging outside or new artwork inside. Digital channels need to evolve just as quickly—a Christmas redesign of your app which isn't ready for release until January is both wasted effort and lost opportunity.

Increasing the velocity of releases is one of the benefits of continuous testing. Once your test system is set up to run tests with every change, your developers are confident that they can push changes faster. They get immediate feedback on whatever new changes they introduce, rather than waiting for human QA later in the development life cycle.

One of the advantages of digital channels in retail is the ease with which a customer can access your products. A retail company should have brand recognition for positive experiences both in store and on their web and mobile applications. All three of these experiences need to be seamless to keep customers returning and increasing their spend. They expect to be able to shop anytime, anywhere, and this is what we strive to give them. Hand-in-hand with this ease of use is the flexibility to move on quickly to a competitor. By having a quality omnichannel experience—from the retail store to the web and mobile applications—your users will continue to be loyal and will allow you to grow your customer base. As mentioned earlier, there are direct correlations to positive user experience and customers spending money with your company. Slow apps, bugs, downtime, and security concerns can drive them away and leave them with a lasting bad impression.

With continuous testing, we can be constantly checking our software for regressions, bugs, and performance problems. Catching bugs early in our process reduces the chance they will make it into production. The confidence that continuous testing brings also allows for faster experimentation, and a tighter feedback loop with the customer as release velocity increases. Overall, a customer experience with fewer bugs and performance and security issues coupled with better reaction to customer feedback is part of the promise of continuous testing.

Software systems are like an old sweater—pulling a thread in one sleeve could lead to a hole in the collar. Changes in software can have unforeseen side effects. For tests to truly bring us protection, they must be comprehensive. They must be repeated often. With each new release,  it is important to implement these checks throughout your pipeline. This repetition allows organizations to guarantee a great user experience prior to release. It also mitigates the need for more traditional human testing, which can be time consuming and creates bottlenecks within your testing process.  So with continuous testing we can both improve the quality of our test runs and free up time to create better quality code to improve the customer experience.

One of the advantages of continuous testing is that it  saves time and resources. In historical software development, the bulk of QA happens once there is a release candidate. The release must then be held up while corrections are made—corrections which may be difficult or impossible due to multiple, concurrent feature changes. By running tests earlier and often, we can detect bugs before bundling new features together. The developer can see that changes are needed, or that a feature needs to be redesigned before investments have been made in other features. Detecting bugs early means that they can be corrected with less effort and without wasting the time of other teams.

Modern retail depends on digital channels, and requires these channels to be reliable. It also requires a constantly improving customer experience, or your customers will go elsewhere. Continuous testing improves the ability to deliver both.

An example of a retail customer who found success with continuous testing is Walmart Labs. Their testing team came to Sauce Labs looking for a quality infrastructure that would also provide support to move from their legacy practices. Between manual testing and  siloed teams, they simply weren’t able to keep up with their online retail demand, which was hurting their business. Once Walmart began their continuous testing process, they were able to go from manually testing to running 50,000+ automated tests per day. Furthermore, they were able to eliminate testing infrastructure maintenance, accelerate their deployment cycles—and save one million engineering hours in 12 months.

If you’d like to see similar testing results for e-commerce, visit our website for resources and information on how retail companies can achieve best practices for continuous testing.