Table of Contents
This guide will walk through the steps of creating an effective deck to share test results with anyone in an organization, from those with no testing experience to the seasoned testing pros.
Step 1: Understand Your Audience
One of the most important parts of sharing test results is understanding your audience, that is knowing who you will be sharing results with and tailoring your approach to speak to them in a way that they will understand.
Presenting a 100 slide deck full of graphs and data to the design team probably isn’t an effective way to communicate your results. That approach probably won’t work with an executive either because they just don’t have the time. But the data-filled 100 slide deck may be just what the analytics team or the developer team wants to see.
The length and content of your deck are two important parts of this equation, but another is the words you choose to use to explain a test. Someone new to testing likely won’t fully understand the common terms testers use every day like CRO, multivariate, significance, sample size, A/B/n, and so on. Those terms should be used sparingly in executive summaries and overviews, opt instead for plain English.
Step 2: Verify Results and Gather Key Performance Data
Verifying test results should be standard practice regardless of whether you plan to share results or not, but it’s especially important when you do. The last thing you want is to share your latest and greatest test with everyone in your organization only to have someone from the analytics team doubt the results, run the numbers, and prove you wrong. Those shake-ups can ruin reputations and destroy confidence in testing, so always make sure your data is good and the results are statistically significant.
After you’re sure the test is significant for your key performance indicators, take the time to gather any other relevant data. People will likely ask questions about channels, demographics, other goals, and other metrics that may not be directly relevant to the test, but are important nonetheless. You want to have that data handy and ready to support your test results because you don’t want to be caught not knowing the answer to a simple question about your test on a department-wide conference call.
Step 3: Craft a Story Based on the Data
The data is important and it’s what CRO and analytics teams work with every day, but it may not be something other teams want to digest. Creating a story that explains the data and describes the motivations, goals, results, and next steps of your tests are key to getting everyone on board and understanding. A great story sells people on the idea of testing and gives you more leeway to continue testing and testing more and different ideas.
Step 4: Create a Results Deck
A great story isn’t any good if you can’t communicate it effectively. The ubiquitous corporate presentation deck is the go-to tool here even with it’s inherent limitations, which can be a good thing. A deck forces you to tell your results story succinctly and without the benefit of just attaching a bunch of XLS data sheets to distract your audience.
Here’s what a great results deck looks like:
- Title slide – the title of the test, date, and responsible stakeholders.
- Summary – give a quick overview of the reasoning for the test, subject, goal, and the result. This slide should be short and to-the-point and include a screenshot of the winner if possible. The goal here is that people should be able to read this slide and understand everything they need to about the test. Everything else just gives more detail.
- Background and Reasoning – explain why you ran the test. Whether it was customer feedback, data analysis, or previous testing that pointed to an issue, give an in-depth explanation.
- Goal – talk about what the main goal of the test was and any other key performance indicators you wanted to increase or monitor.
- Test Variations – include screenshots and quick summaries of the changes in each variation.
- Results Summary – give a quick overview of the results. Was there a winner? What was increased? Did any other KPIs change?
- In-depth Results – give a breakdown of the results for each variation, including the control.
- Next Steps – based on the results of the test, make recommendations for implementing one of the variations live on the site and/or follow-up tests.
Step 5: Share the Results with Everyone
You’ve got your results deck and you’re ready to share with everyone in the company. Before you do though, you need to define the goal of sharing and your purpose.
Here are some goals testing groups may have:
- Get testing buy-in from other business groups that aren’t currently testing
- Get budget increases to hire more testers, get more design/dev resources, buy new testing tools, or head to conferences to increase knowledge and skills
- Get buy-in to test new ideas that wouldn’t be possible without the credibility that previous good results provide
- Prove testing effectiveness to get testing more ingrained in the company culture and involved in strategy, not just test fixes
Whatever your goal is, be sure to keep it in mind as you share results and make your ask right after sharing.
There are several ways to share results, the most common being arranging conference calls. In general, it’s not a good idea to simply send an email blast. Most people don’t read those and you want engagement, people asking questions and getting excited about your test and the results. In person meetings are sometimes the best way to share results and solicit feedback from key players. Getting others involved in your testing goes a long way towards growing a testing culture inside an organization and ensuring continued success.