A team believes a new product will succeed. It interviews friendly customers, demonstrates the prototype to interested partners and collects positive feedback. The results reinforce the original idea. Yet when the company makes a larger investment, the expected demand does not materialise.
The experiment may have been designed to generate encouragement rather than to challenge the assumptions on which the investment depended.
A strong experiment is not a search for approval. It is a deliberate attempt to discover what could make the idea fail.
Confirmation is easy to collect
People often respond positively to a new idea, especially when they are not being asked to spend money, change a process or accept risk. A customer may say that a product is interesting without having a budget, a purchase timeline or the authority to adopt it.
A technical demonstration may show that a solution works under selected conditions while leaving difficult operating environments untested. A pilot may be configured around success criteria that the team already knows it can meet.
These activities are not useless. They become misleading when favourable signals are treated as proof of the most important assumptions.
Identify the failure conditions first
Start by stating the hypothesis in a form that can be challenged. For example: “A defined group of manufacturers will pay a premium for this solution because it reduces a specific source of production loss.”
Then identify what could make it false:
- the loss is not significant enough to justify a purchase;
- the buyer cannot measure the improvement;
- the person who experiences the problem does not control the budget;
- the solution requires integration that customers will not accept;
- an existing alternative solves the problem adequately;
- delivery cost makes the required price uneconomic.
The experiment should seek evidence against these conditions, not simply ask whether customers like the concept.
Choose a test that exposes behaviour
When possible, test actions rather than opinions. Will a customer share process data? Allocate engineering time? Invite procurement into a discussion? Agree to a paid trial? Provide access to a production environment? Commit to a qualification plan?
Behaviour is not always conclusive, but it can provide stronger evidence of intent than general enthusiasm.
For technical uncertainty, test the conditions most likely to reveal failure. For economic uncertainty, measure the outcome and the cost of achieving it. For adoption uncertainty, involve the stakeholders and workflows that will exist after deployment.
Define thresholds before seeing the result
Specify what would count as support, what would count as failure and what remains inconclusive. Without pre-agreed thresholds, teams can reinterpret almost any result as progress.
A failed experiment is valuable when it prevents a larger mistaken commitment or reveals a change needed in the offer. A positive result is valuable when it survives serious attempts to disprove the underlying assumptions.
The strategic implication
Experiments should be designed to improve decisions, not protect internal enthusiasm. Leaders should reward teams for exposing consequential weaknesses early, provided the test was well designed and the learning is used.
The question is not, “How can we prove this idea works?” It is, “What would show us that this idea does not work—and how can we find that out before the cost of being wrong becomes large?”
Related reading
Continue exploring this collection: Resilience — Confluence of Capabilities
Browse all articles · Ideas & Insights
Interested in the decisions behind technology-led growth? Explore Work With Me →
Continue reading in this collection
Preserve Optionality: Stage Gates, Reversibility and Commitment
Staged investment and reversible decisions allow leaders to learn before making large commitments, while protecting the ability to change direction.
Read article →
The Evidence Trap: Why More Data Does Not Always Reduce Uncertainty
More research can create the appearance of rigour without resolving the assumptions that matter. Good evidence is decision-relevant, not merely abundant.
Read article →
What Must Be True Before You Commit Capital?
A capital decision becomes more rigorous when leaders state the conditions that must hold for the investment thesis to work and identify evidence for each one.
Read article →