Winning with a Culture of Experimentation

The business world is transforming before our eyes. Companies are scrambling to identify new ways to engage and convert customers. Good ideas abound, but no one knows for certain what will work. In this environment, testing & optimization have become more important than ever. Companies must do more with less, and the stakes are high. If you haven’t invested in testing, it’s time to start.

Our prior blog on testing highlighted some of the challenges companies face when trying to build a testing program. We’ll now turn our attention to what advocates can do to create a successful program. Stefan Thomke’s recent article in the Harvard Business Review (HBR)[i], ‘Build a Culture of Experimentation’, sets the stage for our discussion. He highlights the need to create a culture that fosters experimentation. The following quote by David Visman, Booking.com’s Chief Product Officer, summarizes the article’s main point. “If I have any advice for [CXOs], it’s this: Large-scale testing is not a technical thing; it’s a cultural thing you need to fully embrace.”

The article provides five prescriptions for creating a culture of experimentation.

1.      Cultivate Curiosity: Management should support experimentation by reducing the stigma associated with failure. Most tests fail to beat the control. Fear of failure inhibits risk-taking, limiting innovation. Companies should create a culture that treats failure as an integral part of testing.

2.    Insist That Data Trumps Opinions: Successful testing cultures embrace experimental findings even when those findings contradict common beliefs. In particular, executives must base decisions on data to foster a culture of experimentation.

3.      Democratize Experimentation: Companies should create a technical infrastructure that allows any employee to launch an experiment. Bookings.com implemented accomplished this by creating a Center of Excellence that provides tools and processes to enable self-service testing and safeguard against catastrophic failure.

4.      Be Ethically Sensitive:  Testing can harm brand equity if consumers believe companies have been unethical. Ethics training and guidelines should be part of employee onboarding and ongoing training.

5.      Embrace a Different Leadership Model:  If democratized experimentation drives decision-making it key areas, it begs question about management’s role in these organizations.  The author argues leaders should focus on setting the vision, providing the resources to enable experimentation, and modeling desirable behavior.

Bookings.com and other large ecommerce business inspire much of Thomke’s thinking, and some of his recommendations only pertain to companies with millions of daily customer interactions. For example, allowing anyone to launch an experiment would not be feasible for companies with low site traffic or that rely primarily on sales through physical locations. However, all companies should heed the basic advice. Culture matters, and leaders must actively cultivate and enable a culture of experimentation to stay ahead of the competition. In subsequent blogs, we will explore what kinds of enablement are required at the people, process and technology levels.

[1] Thomke, Stefan. ‘Building a Culture of Experimentation’.
Harvard Business Review (March-April 2020) Vol. 98, pp. 40-48.

‍