A lot of digital marketers and paid specialists are running A/B tests. But few are testing strategically, driving growth and insights from every single experiment. And fewer still are pushing an experimentation-based cultural shift at their companies.
But that’s where the money’s at. A culture of experimentation helped Microsoft’s Bing unit boost revenue per search by 10% to 25% a year. It’s what powers the success of digital leaders like Amazon, Uber, Netflix, Airbnb, Booking.com, and Expedia.
So why do so many companies struggle here? Because getting buy-in for experimentation as a strategy is hard. Embracing “failure” as a path to growth is hard. Driving cultural change is hard. Stefan Thomke explains in Harvard Business Review:
“As companies try to scale up their online experimentation capacity, they often find that the obstacles are not tools and technology but shared behaviors, beliefs, and values. For every experiment that succeeds, nearly 10 don’t—and in the eyes of many organizations that emphasize efficiency, predictability, and “winning,” those failures are wasteful.”
But — as the leaders mentioned above will tell you — it’s worth it. So here are three tips to help you go from running the occasional A/B test on ad copy or a paid landing page to fostering a revenue-driving culture of experimentation at your company.
On April 21st at 2pm, join conversion experts from AB Tasty and ROI Revolution as they unravel the complexities of elevating your customer experience through conversion rate optimization. In this live webinar, you’ll uncover:
- Key personalization tactics to help you stay ahead of changing consumer behaviors.
- How to structure your optimization strategy to fuel brand growth.
- 3 ways to improve the user experience on your website.
Optimizing your website for conversions takes time, effort, and the right strategic partners. Register today for this webinar to kickstart your optimization journey.
3 Tips to Start Building a Culture of Experimentation at Your Company
These tips are things you, as an individual, can start doing right now to develop your company’s culture of experimentation. Consider this a jumping off point.
1. Make Sure There’s a Method to Your A/B Tests
The right tools make it pretty easy to run A/B tests. You can test your ad creative and copy. You can test your landing page imagery, headlines, and structure. But if you want to get actionable results from every test, you need to test strategically.
First, make sure you’re prioritizing tests on your highest impact digital properties. There are lots of frameworks out there for test prioritization — like PXL, PIE, ICE, and more — but they’re generally assessing similar things:
- How bad is this digital experience? (How much potential is there for improvement?)
- How valuable is this digital experience? (Is it a high-traffic page? Are you sending a lot of paid traffic there?)
- How difficult will it be to test on this digital experience? (Do a lot of stakeholders need to approve a test on this experience?)
You can use one of the frameworks above to evaluate where you should test first and plan your testing roadmap so you can ensure maximum impact.
You should also make sure you’re building your tests around a solid hypothesis. A strong A/B test hypothesis should be based on evidence (insights from past tests, customer research, observed audience behavior, examples from competitors, etc.). It should include:
- The change you plan to make
- The conversion goal you’re hoping to impact
- The reason you think this change will have an impact
For example: Changing the headline on paid landing page X to mirror the copy on ad X will decrease bounce rate because visitors will feel assured that they will find what they’re looking for on the page they’ve landed on.
When your tests are properly prioritized and based on robust hypotheses, they will always yield learning if they don’t drive uplift. This means you can show value from every single test and clearly communicate your next steps, which means you can build good will and acceptance around experimentation as a broader strategy.
2. Socialize Your Experiments in the Right Way, to the Right People
This brings us to tip #2, which is all about communication. It can be tempting to keep quiet about the tests you’re running, especially considering most won’t drive big shiny uplift. But don’t fall into the trap of only sharing winning results. Because eventually you will hit a plateau and stakeholders will start to doubt the credibility of testing as a strategy.
Transparency is critical when you’re trying to build a culture of experimentation. But that doesn’t mean you can’t tailor your communications to your audience. In fact, you should.
You may want to communicate consistently and in detail with the individuals close to the experiments: the designer, researcher, web developer, program manager. Anyone who worked on the experiment should be kept in the loop so they can see the impact of their work. Your internal messaging tools, like Slack, can be a great space for this.
When you’re communicating to a manager or more senior stakeholder, you’ll want to package up the most important details in an easy-to-consume way. Include high-level details like your test area and hypothesis, the results, and next steps.
Hot tip: Presenting multiple test results to stakeholders on a recurring basis will help them see your experimentation efforts as a program rather than ad-hoc. Use this time to demonstrate the power of iteration: How learnings from one test can inspire another test, ultimately driving winning results.
You can also consider communicating with a broader team or your full company to build excitement for testing in other departments. Keep in mind: This may inspire people to share ideas or want to start testing (which is great), but make sure your company has the resources to scale testing strategically.
3. Lay the Groundwork for Testing at Scale
It’s never too early to think about experimentation program management. I know, it’s not the flashiest tip — but it will allow you to scale testing and set you up as your company’s experimentation champion. You can start with a couple of basic documents: an experiment strategy document and an experiment tracking spreadsheet.
An experimentation strategy document might include…
- Long-term objectives for your experimentation program tied to your company’s business objectives.
- Shorter-term SMART goals for testing.
- Experiment requirements (like evidence and a hypothesis) and the steps to launching an experiment.
- Roles and accountabilities: What skills do you need to design and launch an experiment? What is each person responsible for? When do you need stakeholder approval?
- Communication guidelines: How should someone socialize experiments to the team and key stakeholders?
- Experimentation ethics: Be explicit about experimentation dos and don’ts, particularly around data privacy and persuasion tactics.
An experiment tracking spreadsheet is where you can begin to build an archive of tests, test areas, test hypotheses, results, and learnings. This archive will become a source of evidence for future tests for both your team and other teams as you scale testing.
Hot tip: Ask your A/B testing tool vendor if they have features that support building an experiment database.
Key Takeaways
Companies that nurture a culture of experimentation are leading the pack. If you’re running A/B tests on your paid campaigns, you have the opportunity to build your company’s culture of experimentation. You can be the champion who helps your company embrace testing and make major gains in the short-term and the long-run.
It starts with a few simple actions:
1. Evaluate how you’re running A/B tests right now and make sure you’re prioritizing your highest impact areas and writing hypotheses that result in insights.
2. Figure out how to communicate the impact of your testing program to key stakeholders (use the channels and methods that work for your unique company culture).
3. Build infrastructure in the form of key strategic documents that will allow you to scale experimentation across other teams.
