The “Lean Startup” and Why It Is Killing Growth Experiments

Over the past few years, I've seen the "Lean Startup" grow to biblical proportions in Silicon Valley. It has introduced a lot of clever concepts that challenged the old way of doing business. Even Enterprises such as GE, Intuit and Samsung are adopting the "minimum viable product" and "pivoting" methodologies to operate like high-growth startups. However just like any dogma, the "lean startup" when followed with blind faith leads to a form of obscurantism that can wreck havoc.

Understanding "activation energy"

A few weeks ago, I was discussing implementing a growth experiment with Guillaume Cabane, Segment's VP of Growth. He wanted to be able to pro-actively start a chat with Segment's website visitors. We were discussing what the MVP for the scope of the experiment should be.I like to think of growth experiments as chemical reactions, in particular when it comes to the activation energy. The activation energy is commonly used to describe the minimum energy required to start a chemical reaction.

The height of the "potential barrier", is the minimum amount to get the reaction to its next stable state.In Growth, the MVP should always be defined to ensure the reactants can hit their next state. This requires some planning which at this stage sounds like the exact opposite of the Lean Startup's preaching: "ship it, fix it".

The ol' and the new way of doing

Before Eric Ries's best seller, the decades-old formula was to write a business plan, pitch it to investors/stakeholders, allocate resources, build a product, and try as hard as humanly possible to have it work. His new methodology prioritized experimentation over elaborate planning, customer exposure/feedback over intuition, and iterations over traditional “big design up front” development. The benefits of the framework are obvious:- products are not built in a vacuum but rather exposed to customer feedback early in the development cycle- time to shipping is low and the business model canvas provides a quick way to summarize hypotheses to be testedHowever the fallacy that runs rampant nowadays is that under the pretense of swiftly shipping MVPs, we reduce the scope of experiments to the point where they can no longer reach the "potential barrier". Experiments fail and growth teams get slowly stripped of resources (this will be the subject for another post).

Segment's pro-active chat experiment

Guillaume is blessed with working alongside partners who are willing to be the resources ensuring his growth experiments can surpass their potential barrier.The setup for the pro-active chat is a perfect example of the amount of planning and thinking required before jumping into implementation. At the highest level, the idea was to:1- enrich the visitor's IP with firmographic data through Clearbit, 2- score the visitor with MadKudu3- based on the score decide if a pro-active sales chat should be promptedSeems pretty straightforward, right? As the adage goes "the devil is in the details" and below are a few aspects of the setup that were required to ensure the experiment could be a success:

• Identify existing customers: the user experience would be terrible is Sales was pro-actively engaging with customers on the website as if they were leads
• Identify active opportunities: similarly, companies that are actively in touch with Sales should not be candidates for the chat
• Personalize the chat and make the message relevant enough that responding is truly appealing. This requires some dynamic elements to be passed to the chat
• ...

Because of my scientific background I like being convinced rather than persuaded of the value of each piece of the stack. In that spirit, Guillaume and I decided to run a test for a day of shutting down the MadKudu scoring. During that time, any visitor that Clearbit could find information for would be contacted through Drift's chat.The result was an utter disaster. The Sales team ran away from the chat as quickly as possible. And for a good cause. About 90% of Segment's traffic is not qualified for Sales, which means the team was submerged with unqualified chat messages...This was particularly satisfying since it proved both assumptions that:1- our scoring was a core component of the activation energy and that an MVP couldn't fly without it2- shipping too early - without all the components - would have killed the experimentThis experiment is now one of the top sources of qualified sales opportunities for Segment.

So what's the alternative?

Moderation is the answer! Leverage the frameworks from the "Lean Startup" model with parsimony. Focus on predicting the activation energy required for your customers to get value from the experiment. Define your MVP based on that activation energy.

Going further, you can work on identifying "catalysts" that reduce the potential barrier for your experiment.

If you have any growth experiment you are thinking of running, please let us know.
We'd love to help and share ideas!

‍Recommended resources:
HBR [1]
HBR [2]
Venture Beat
HeavyBit‍‍
‍
Images:
Fake Grimlock
Britannia
Infoq
Wikipedia‍‍