Testing Effectively with Limited Resources

One of the biggest challenges for any development organization is managing time and development resources effectively. Given that software testing (if you’re doing it thoroughly, which you should be) is one part of the delivery pipeline that consumes a great deal of resources, finding ways to make software testing more efficient is one useful strategy for improving overall resource consumption.

With this goal in mind, this article discusses best practices for improving your testing model. The main goal of the tips below is to suggest ways in which you can increase your test coverage while reducing the time, infrastructure and effort you spend on testing.

As software organizations have evolved, the timeline for the development lifecycle has seemingly shrunk. Today, we have to deliver software on a rapid, “continuous” basis. With this increase in the speed of development comes challenges as they relate to ensuring a high level of software quality within a tight timeframe.

While writing a bunch of automated test scripts and running them one after another as part of your build process is a good way to begin addressing this challenge, it may not result in great efficiency or adequate test coverage for your application. It’s more likely that some strategic measures have to be taken to ensure that features are tested thoroughly and that time is not wasted, both during your testing routines and as a result of your testing routines.

We know that we need to perform application testing in such a manner as to not slow down the speed of delivery by saving the engineers’ time. That being said, we need to employ strategies that also improve test coverage and lead to faster bug discovery. Consider the following two testing strategies:

One way in which you can improve test coverage while saving time is to run test scripts in parallel. Consider the value in running tests in parallel for a web application. Instead of running one test script for each browser and OS combination in a sequential manner, you can run several test scripts for many browser and OS combinations simultaneously. For an organization that runs a high volume of tests with each application build, this can be invaluable in two specific ways.

The first notable benefit of parallel testing is simply the increase in the velocity with which the application testing will occur. Keeping the testing routines manageable in this manner will prove critical in ensuring that engineers’ time can be used more effectively by getting them back into development quickly after verifying the success of the build.

Secondly, parallel testing leads to a massive increase in test coverage, even while testing velocity increases. This is especially true when leveraging a cloud testing infrastructure where a large number of VMs allow for a greater number of tests to run in parallel. Rather than needing to pick and choose browser and OS combinations against which you will run your scripts due to the time constraints inherently imposed by running them sequentially, you can run those same tests for essentially all available browser and OS combinations in a matter of minutes. This way, you are ensuring a high level of software quality for virtually all users of your application, regardless of browser or operating system. And it should be noted that web application testing in parallel using the Selenium framework is made even easier with the Sauce Labs testing grid, where over 700 browser/OS permutations are available for use when testing your application.

Headless testing is another option for increasing testing speed while ensuring a high level of confidence in an application’s quality. To understand what is meant by headless testing, we must first understand the meaning of a headless browser. Headless browsers are essentially web browsers that are not equipped with a GUI. When considering the effort to increase testing performance, headless testing should be considered due to its lack of a GUI. The overhead of rendering the web page is essentially eliminated, allowing the tests to simply run in a lightweight manner and complete more quickly, providing almost instantaneous results on application quality. Take a look at Sauce Headless for more information on Sauce Labs venture into the world of headless testing.

Due to its lightweight nature and ability to return quick results, headless is a useful solution for testing an application early on in the development process. This idea of testing early in the development process has a name — It is known as shift-left testing.

I’d be remiss not to mention the benefits of shift-left testing in the effort to improve testing strategy. Shift-left testing refers to the effort to implement testing as early as possible in the development pipeline. When testing begins later in the development cycle, it is more likely that more impactful bugs will be found toward the end of the process. This can make them more expensive to resolve, and could lead to the release schedule being upended.

If these bugs are discovered at a point much earlier in the process, then it’s likely the fix will be simpler and easier to implement, saving developers’ time and other resources down the line. In this way, shift-left testing serves to minimize a lot of the risk and uncertainty in your development schedule. By the time you enter the later stages of the process, you will most likely be dealing with minor bugs rather than major ones.

A high level of software quality can be ensured when a testing strategy is implemented that is both efficient and effective. A huge part of determining whether or not a testing model is efficient is evaluating whether or not the test strategy operates effectively within the time constraints of the release schedule, without the need for the organization to expand upon its current staff to meet its testing needs. Test implementation strategies, such as parallel testing and headless testing, along with high-level concepts such as shift-left testing are effective agents in adding to both test volume and test coverage, while saving time and money for the organization.

Scott Fitzpatrick is a Fixate IO Contributor and has 7 years of experience in software development. He has worked with many languages and frameworks, including Java, ColdFusion, HTML/CSS, JavaScript and SQL.