In software testing, running the tests is the easy part. Communicating the results of tests to all stakeholders is what’s challenging.
That’s especially true in today’s age of DevOps, which places a priority on continuous communication and continuous improvement — goals that you can only achieve when you have an effective way of sharing software testing data across your team.
With that challenge in mind, I’d like to take a look at strategies for sharing software testing information with all stakeholders—whether they are other test engineers, developers, admins or even non-technical staff who might have a role to play in application delivery and quality.
I’ll focus on the concept of using a communication template to achieve the information sharing that teams today require.
Your Software Testing Communication Template Starts with Setting Goals
The goal behind the communication template for testing results is to provide insight, accountability, and awareness for teams. If you're seeking a cohesive way to view and communicate the state of quality across the organization better and faster, you're in the right place.
The intention of the communication template is more than a voice in a meeting, an email or sending a message in a Slack channel. It is a visualization of the organization — how the organization is performing from a quality standpoint, as well as the metrics that will influence how to drive quality improvements forward.
Drafting the Software Testing Template
We live in a world full of data. Why do we need to manually sign off or relay information regarding the state of quality? How do we take the traditional test reporting from doom-and-gloom experience to the realm of data analytics, allowing us to draw conclusions and make decisions quicker from test metrics? Achieving this transformation requires a strategy, clearly defined acceptance criteria, tooling, documentation, easy access, awareness, and communication.
Let me walk through the steps I’ve taken to improve our communication for quality metrics.
Software Testing Communication Strategy
A communication strategy is essential for any project because it provides a sense of direction and outlines measurable goals. The aim is to remove the headache of manual collection of testing results, defect lists, and bug reports in various locations. I wanted to create a solution that visualizes testing results and other quality metrics from a single-pane view with easy access for all different types of users. We can start relying on data we are generating from the continuous integration (CI) pipeline, GitHub checks, and defect tracking and monitoring to act on the data presented to the team.
As you’re developing your strategy, these are some of the questions you’ll want to answer:
Do you want to troubleshoot test failures?
What is the quality of testing in various environments (dev, staging, prod)?
Are we catching failures in lower environments?
Is the data the same in dev and pre-production environments?
Is there real-world data to be used as a base? (And is data masking required?)
What is the health of particular applications?
How many bugs are being created for teams and applications?
Is the number of tests being executed trending up or down?
How effective is the release cycle?
Are defects trending up or down?
What is your pass/fail rate?
What is the level of efficiency from executing tests in parallel?
Test Acceptance Criteria
Unclear acceptance criteria leads to asking: “Did we build the right product?” We want to build a dashboard that can paint a single view and can be utilized by executives, managers, developers, quality assurance, DevOps, and others. It should include and/or provide:
Global access for ease of use
A clear understanding of the test metrics you want to collect (e.g. pass/fail, skip, number of active tests, individual test execution times, and more)
Increased awareness and accountability
The ability to use it as a knowledge and training tool to help improve quality
Improved communication through a single source
Views into the quality state of the application through testing data collected from CI/CD pipelines, monitoring, and more
The ability to show testing results, while allowing stakeholders to filter results
Testing results and trends from the latest pull request (PR), or over a period of time
A method of reporting quality that automatically drives a faster feedback loop and quicker turnaround time on decision-making to determine the criticality level of any failures
Communication of all critical details
Tooling for Software Testing Data Communication
So many options are available when it comes to tools you can use to collect and share software testing data. I’m not going to spend any time here on the available options, but your tooling needs to collect quality metrics from various sources continuously, and ingest the metrics to be managed by a dashboard application that allows you to analyze all the metrics to make better decisions on the health of the application.
Awareness
Focus on awareness and building the right level of communication of testing results, no matter the quality of your product and service, no matter how much energy you expend advocating your movement for visualization of quality metrics. As an organization, we need to proactively share value stories from quality dashboards, actively soliciting feedback and recommendations, and facilitate communication between multiple stakeholders.
Conclusion
Effective communication of test results allows stakeholders to view the test data and visualize how the organization or team is performing from a quality standpoint. It will lead to building a continuous improvement culture.
...
Greg Sypolt, Director of Quality Engineering at Gannett | USA Today Network, maintains a developer, quality, and DevOps mindset, allowing him to bridge the gaps between all team members to achieve desired outcomes. Greg helps shape the organization’s approach to testing, tools, processes, and continuous integration, and supports development teams to deliver software that meets high-quality software standards. He's an advocate for automating the right things and ensuring that tests are reusable and maintainable. He actively contributes to the testing community by speaking at conferences, writing articles, blogging, and through direct involvement in various testing-related activities.
