In the age of DevOps, virtual machines, continuous delivery, and microservices, is end-to-end testing necessary? Is it even viable?
The short answers are "yes" and "yes". But don’t just take my quick word for it. Let me explain why end-to-end testing is crucial for effective DevOps.
Defining End-to-End Testing
Before we consider what value end-to-end testing brings to an organization, let me clarify exactly what I mean when I speak about end-to-end tests.
End-to-end testing is the process of testing your entire system from start to finish. The purpose of this is to ensure that information is passed between components within the system in the correct manner, and that the integrity of the information is maintained through the process. End-to-end tests are usually written from the perspective of the end user, and are typically automated through frameworks like Selenium and Appium.
Benefits of End-to-End Testing
There are many benefits to both writing and regularly executing end-to-end tests as part of the deployment process. These include, but are not limited to:
Requirements Validation
Regression Testing
Validation of Environment Configuration
Validation of Functionality
Consider a common application of technology on the Internet—that of an eCommerce site. The basic requirements: a user would be able to browse products, log in to the system, add a product to a shopping cart, and pay for the product, before finally receiving a confirmation of the sale, and subsequently being shipped either a physical product, or gaining access to a digital product.
Additional requirements of this system might include the ability to purchase multiple products, or a mix of digital and physical products. The payment requirements might include credit card processing, Paypal, gift cards, or a combination of those methods.
By developing a series of end-to-end tests which validate each of these scenarios, the organization can confirm that each is configured correctly, and that the requirements of each have been met. In addition, the organization can be reasonably certain that a customer performing any of those steps can expect the application to function appropriately.
End-to-End Testing and the Engineering Team
In most large software development environments, a software solution is likely to be a series of components, each developed and maintained by a different team. From the perspective of the team, there is limited value in writing a suite of all-encompassing end-to-end tests. While they may be useful in identifying malfunctioning components of a system, they will generally only point out the failure of a component in general terms, and those tasked with fixing the issue will need to investigate further to identify the exact cause of the bug.
For these engineers, the most value can be gained by ensuring the individual functionality within their code is covered by unit tests. If properly written, a unit test will help an engineer quickly identify problems as well as enable the rapid identification of exactly which functions contain the offending code.
To this end, when a bug is uncovered by an end-to-end test, the resolution to the bug should include not only the fix, but also a series of unit tests to prevent the bug from being introduced back into the system when future updates are made.
If Not Engineering, Then Who?
The primary benefactors of end-to-end testing are those members of your organization who are concerned with the business aspects of the solution. They might include product owners, management, and stakeholders in the specific products being developed. Specifically, these positions are concerned with ensuring that system requirements have been met, and that the system is usable and reliable for customers of the solution.
Additionally, well-written and comprehensive end-to end tests provide confirmation that the system as a whole is functioning as intended and that the environment has been correctly configured. Ultimately, the result of comprehensive and regularly executed end-to-end tests is an increase in confidence about the stability of the application.
A Vital Component
So while the case may be made that end-to-end testing doesn’t present a great deal of value to the engineering and development teams in an organization, a strong case still exists for the inclusion of well-written and comprehensive end-to-end tests. They provide value to the business-focused aspects of an organization, and ultimately function as the glue which ensures that configuration, and integration of application components within the environment, are functioning as expected and meeting the expectations of the organization, and more importantly, the end user.
Want to see where your organization falls in its DevOps Journey? Take this brief DevOps Assessment Quiz to find out.
Mike Mackrory is a Global citizen who has settled down in the Pacific Northwest - for now. By day he works as a Senior Engineer on a Quality Engineering team and by night he writes, consults on several web based projects and runs a marginally successful eBay sticker business. When he’s not tapping on the keys, he can be found hiking, fishing and exploring both the urban and the rural landscape with his kids. Always happy to help out another developer, he has a definite preference for helping those who bring gifts of gourmet donuts, craft beer and/or Single-malt Scotch.
