Winning with saucectl, Part 1

In case you haven't yet taken saucectl (say "Sauce Control") out for a test drive, allow me to convince you that you should.

If you are reading this, you are probably a busy test engineer, or an application developer seeking to test new features reliably. You've already realized you have no hope of efficiently re-creating even a small portion of what Sauce Labs offers, and you're wise enough not to try. Instead, you stay focused on your core business. Meanwhile, Sauce Labs enables you to deploy to your customers more regularly and with higher confidence.

Sauce Labs grew up with Selenium in the early days. As a test engineer, I, like many of you, have grown up with both Selenium and Sauce Labs. Since then, Sauce Labs has grown into what I have described as a "TaaS" platform: testing as a service. Through both acquisitions and in-house development, Sauce Labs now offers a growing list of testing capabilities including functional regression (its roots), visual regression, API testing, performance, and of course mobile. saucectl is just one recent addition and the subject of this article.

When I began working in test engineering, there were no tools that we would appreciate today. A very few tools were expensive, inflexible, and questionable in terms of their net value added when all was said and done. The tools themselves required much investment on our part to purchase, install, configure, and maintain. Certainly we were not seeing anything that positively improved software development as a discipline.

Except for a season when thin clients enjoyed a brief resurgence, platform diversity was the norm. That increased the lift for those early tools. Vendors had little motivation to write new code for another OS. That was expensive, and they had the leverage. They could just say no. 

This didn't create, but undeniably contributed to, the siren cry of, "But it works on my machine!" That nails-on-chalkboard phrase became so ubiquitous that it made its way onto shirts and mugs. In time, people recognized it as the barrier that it was, and began solving it where they could, one piece at a time. Those incremental improvements came to the test engineering space as well.

In saucectl I see a solution to that awful problem.

Shortly after I created a new repo for a test drive of saucectl, I came to appreciate its contribution to velocity and quality. Let's talk about velocity first. If I can make it easier for anyone on my team to run my tests, then my tests will be run more often. Problems can be found sooner. Features can be tested immediately, at any hour, and that's no small thing for a team spanning time zones. And soon, the team will be able to deploy without me if they need to. I'm thrilled. I can't be present for every deploy at every hour, and I don't want to hold us up. 

As for quality, a fresh set of eyes can't hurt. What if I've missed something in my test design? What if my tests aren't really doing what I think they are? The tests themselves are code. QA is programming. We are all programmers now. And we all create bugs. I'm the only test engineer on a team with a bunch of kick-butt coders. They deliver. A lot. I'm busy. I can't catch everything. Anything I can do to make the tests themselves more "just part of the code" is worth doing. 

Sure, we do PRs. PRs are necessary but not sufficient. With saucectl I'm advancing test development as a team activity. I'm enabling a culture of greater teamwork. I’m giving the developers a way to become more involved in the testing of their own code and mine. The accuracy and efficiency of our testing is going to improve. 

On our team, as on yours no doubt, each developer has a specialty. (If your manager was good, she recruited with that in mind.) When test design becomes less about what one test engineer does and more about what a team can do, with each one adding their specialty to the mix, our testing can really move forward. 

For years now, I've watched that artificial line between "dev" and "QA'' get rubbed out. I’ve never tried to stop it. Indeed, I've done my part to accelerate this trend. Lately, I've done that with saucectl.

saucectl has been a huge win for us. It has been one of the biggest leaps forward in test efficiency that we've had in recent years.

In this post, I have made a case for saucectl, and hopefully persuaded you to at least try it. I would remind you, in closing, that you could be up and running with your own proof of concept in less than a day and at no extra cost. You don't have much to lose but much to gain.

In the next post, I share with you some more technical details and some lessons I learned from my own POC.

Allen Loew is a Principal Quality Engineer at Progressive Leasing LLC, and a Sauce Labs advocate. In support of his current team, he has written a library of Playwright tests in JavaScript and running in saucectl. Previously, he wrote most of his tests in Java and Selenium. Allen was an early adopter of Sauce Control and is now all-in, exploring boundaries of efficiency with sharding and multithreaded test execution. Allen has championed Sauce Labs usage at his current and former employers, and has led training efforts for new users. His particular interests in testing include multi-platform coverage, concurrency, and localization.