The Sauce Labs blog previously featured the post “Headless Browser Testing 101,” covering the fundamentals of headless testing, its advantages, and the purposes for which it is used.
This post takes a deeper dive into a specific aspect of headless testing: How it fits into a Kubernetes-driven CI/CD pipeline. We’ll focus on how containers and Kubernetes change testing and what you need to change in your testing routine to accommodate containerized applications.
The impact of containers on testing
The goal of DevOps is to ship high quality software faster. There are many cloud-based testing solutions and open source testing tools that have brought teams closer to realizing this goal more than ever before. Yet, there is always room for improvement. In recent times, we’ve seen the container revolution sweep the world of software development. Testing has undergone tremendous change as well.
Particularly, containers have made it possible to run tests on lightweight instances in the cloud. Rather than waiting almost a minute for VMs to initiate and be configured, containers can be spun up in seconds, or in some cases, in a fraction of a second. This means there’s even greater opportunity to quicken testing.
QA should find ways to leverage this opportunity. One way of doing this is to give headless testing a more prominent place in the QA process. Headless testing and containers go well together as they are fast, resource-efficient, and nimble. But where in the pipeline is the most need to quicken testing? Right at the start, of course.
GitOps requires headless testing
Continuous Integration typically kicks in after a developer commits code. The CI process focuses on automating the build and test parts of the pipeline. This could be considered as the middle section of the development pipeline. Test automation solutions like Sauce Labs have catered to these needs for a long time.
What’s changed with the advent of containers and Kubernetes is the emergence of GitOps, which I see as version 2.0 of continuous integration. GitOps takes CI to its fullest extent, stressing the importance of automated processes even more, and bringing this automation even earlier in the development cycle.
With GitHub playing a central role in the way DevOps teams create and collaborate on code, it has become the starting point of the GitOps process. Unlike before, when manual authorization by QA, SecOps, or the release engineer was the final step before releasing code, today, GitOps promotes complete automation of this process end-to-end. However, this results in some conflicting priorities. While teams want to ship code faster and completely automate the process, they don’t want to compromise on security, quality, and reliability. This is where headless testing comes in. The best way to bake these priorities into development is to test early, and test often.
Headless testing enables testing during development. Traditionally, testing during development has not been popular because testing is seen as a distraction rather than a facilitator of development. However, headless testing makes it possible for developers to run a test in their development environment and get feedback in a few seconds. This is extremely valuable as it doesn’t slow down development. And importantly, it helps developers write better code.
Pre-commit testing for GitOps
Within the GitOps movement, there’s the trend of having a “dry run” before every “merge.” Tools like Jenkins X and Weave Flux endorse this approach. However, the challenge with executing dry runs is that it’s not easy to test on every pull request. That’s too frequent and heavy for traditional automated tests. Headless testing fills this gap as it can enable dry runs in the form of pre-commit testing.
When running tests this early in the pipeline, there isn’t any time for manual testing. Headless testing is ideal here because it has a lot of scope for automation. You can automate a series of actions that make up a test use case, have each step recorded in the form of screenshots, and receive the final output of the test.
Not just post-release tests
Pre-commit, headless testing doesn’t only have a place in post-release testing. It can be used to run sanity checks to verify the functioning of particular parts of the application, or even record a timeline of changes to any part of the app.
Headless testing is meant for running functional tests, and integration tests where a person doesn’t need to see the test running. However, during the process, some tests that fail may require further testing and closer review of the features under test. In these cases, headless testing can be used effectively along with traditional testing. While headless testing provides screenshots, traditional automated tests can give you videos of each step with more details. By using this combination, you can drive greater efficiency in your overall testing strategy.
In conclusion, headless testing has long been overlooked as a hack-job to generate screenshots of applications automatically. However, with the advent of Kubernetes and GitOps, headless testing is taking on a new prominence in the CI/CD pipeline. By enabling testing early in the pipeline, and making tests faster, headless testing helps ship quality code faster.
Twain Taylor is a Fixate IO Contributor and began his career at Google, where, among other things, he was involved in technical support for the AdWords team. His work involved reviewing stack traces, and resolving issues affecting both customers and the Support team, and handling escalations. Later, he built branded social media applications, and automation scripts to help startups better manage their marketing operations. Today, as a technology journalist he helps IT magazines, and startups change the way teams build and ship applications.
