If you work in DevOps, you probably already know why continuous testing is valuable. Continuous testing, which refers to the use of automated tests whenever possible within the application delivery pipeline, provides end-to-end automated testing that leads to higher application quality while preserving the delivery schedule. (It can help improve security and performance, too, as automated tests also address these areas.)
But the question for many DevOps professionals today is: Just how far have we come in embracing and implementing continuous testing? Which continuous testing tools and strategies are available today? And what can we do as DevOps engineers to take greater advantage of continuous testing and test automation?
This article answers those questions by surveying the state of continuous testing within the DevOps world today. I'll discuss the basics of continuous testing in DevOps organizations, as well as some tools currently available for implementing a continuous testing strategy. We’ll then get into what we’ve learned so far regarding continuous testing in DevOps, and then address a question very familiar to those utilizing the DevOps methodology — Where do we go from here?
Continuous Testing in DevOps: What Have We Learned?
As software testing has evolved, processes have been created to improve an organization’s ability to diagnose application issues earlier. Continuous testing is one of these processes. Here is how it works:
Automated tests are produced to thoroughly test software features as they are integrated into the code base.
These tests are then integrated into the continuous delivery pipeline. This can occur by hooking these automated tests up through the continuous integration tool used by the organization. In this manner, the automated tests will run each time changes to the application code are committed to the code base.
If an issue arises with an application feature that has implemented automated testing, then it will be caught as code modifications are committed to the code base. In doing so, bugs within the application code are caught as early on in the development process as possible, thus saving time for developers later in the development process.
Throughout the evolution of software testing and the DevOps methodology, we’ve learned that bugs are less expensive to fix when they are found earlier in the development process. The longer it takes to identify an issue with the code, the more likely it is that other code within the application depends on this erroneous code, leading to a more time-consuming fix (as the bug may be more far-reaching). So in order to achieve this level of early bug detection, automated testing tools were developed to become the backbone of a continuous testing strategy.
Automated Testing Frameworks: Making Life Easier
With the embrace of continuous testing by the software testing community, automated testing frameworks have become an integral part of a continuous testing strategy. These frameworks make the life of a software tester easier by enabling them to simply develop test scripts that thoroughly validate the code in an efficient manner, and the scripts can be run at any point in the development process.
Selenium
Selenium is an automated testing framework for web applications that allows for the development of test scripts that serve to automate browsers. These scripts can be easily modified if the feature being tested should change, and Selenium supports the development of tests in many languages to ensure that you have the ability to develop your test suite using a language in which you are comfortable developing.
For added efficiency, these tests can be run in parallel in multiple browsers utilizing a cloud platform such as that offered by Sauce Labs, thus ensuring code quality while saving time and supporting continuous delivery.
Appium
The tremendous rise in development of mobile applications has led to the need for automated testing tools for mobile applications. Appium is essentially that. An open source testing framework for mobile applications, Appium is designed for use with native, hybrid and mobile web applications.
In addition, Appium’s role in continuous testing may be even more critical than that of tools for continuous testing with traditional web applications. This is due to differences in deployment strategies between these application types. While a traditional web application can be deployed at virtually any time, the same cannot be said for mobile applications. Restrictions on deployment enforced by the app stores, which essentially serve as gatekeepers, mean that it may take longer to re-deploy a mobile application.
This raises the stakes for developers and testers to “get it right” prior to a production deployment. Otherwise, it’s possible to deploy code which contains bugs to a production environment that cannot be resolved in a timely fashion. So it’s easy to see where a tool like Appium can come in handy for a DevOps team eager to implement a successful and airtight continuous testing strategy.
Opportunities for the Evolution of Continuous Testing
With the growing need for DevOps teams to get code out the door quickly and efficiently, the importance of continuous testing also continues to grow. And as it continues to grow, it continues to evolve. One possible direction for this evolution is towards AI test automation.
AI refers to machine learning, and the hope in the software testing world is to create tools that utilize machine learning to identify potential test cases and generate code automatically that can be used to effectively test features within an application. This would significantly reduce the human involvement in test creation and help to close gaps in test coverage, leading to true end-to-end automated test coverage — and hopefully, as a result of AI automated testing, developers and testers alike will be able to sleep a little more soundly knowing that their code has been tested as thoroughly as possible, and that bugs are being discovered as early in the process as possible, giving the developers all the time they need to resolve issues and meet delivery deadlines.
Conclusion
Continuous testing refers to the software testing strategy involving end-to-end automated testing. It assists DevOps organizations in the early detection of software bugs, allowing for remediation of issues at an earlier part of the development lifecycle. As this strategy has evolved, innovative frameworks and tools have been created and advanced in an effort to improve test coverage, while limiting manual intervention in software testing along the way.
Scott Fitzpatrick is a Fixate IO Contributor and has over 6 years of experience in software development. He has worked with many languages, including Java, ColdFusion, HTML/CSS, JavaScript and SQL. Twitter: @sc_fitzpatrick
