The Hypothesis Driven Startup

Founders must be almost irrationally optimistic about their prospects. It is optimism steels them against rejection and hard times. However, founders must also be realists and think objectively about their business.

They can, and should, dream of the ways it could work but they must not lose sight of the risks lurking around the corner. One way to avoid this common pitfall is to treat the early stages of every idea, be it for a business, for a marketing push, or a new product as a series of hypotheses to be tested.

Why should founders be hypothesis driven?

  1. It’s harder to lie to yourself about progress - this is a massive risk for early stage founders.
  2. It focuses the mind on the main thing - the main thing is the hypothesis or part of the hypothesis to be tested.
  3. It takes emotion out of decisions and makes it easier to say no - there is no room for projects that aren't going to work. Time, focus and motivation are scarce resources.
  4. It helps guide the ideation flywheel - “Failure” is not only an option, it’s an expected result that can be appropriately framed as the launch pad for the next iteration of your idea.
  5. It gives teams solid success metrics before traditional metrics (active users, revenue, churn etc) are possible.
  6. It imbues a culture of ambition and accountability - first to yourself, then to your investors, then you to your employees and vice versa. Focusing on solid success metrics aligns the entire team on priorities.

A practical guide to Hypothesis Testing

1. Stack rank your risks

What is the thing that is most likely to do the most damage to your idea? This is the thing you need to be focused on first.

It’s very natural to shy away from this. No one likes to think about killing their baby, however necessary it might be. This, above all else, is the pain you need to face up to. The psychological pain of being misguided, of having poor intuition. With that said, you can’t test every last thing - there will always be unknowable or untestable risk to face - this is the gamble founders take.

2. Run headlong into the biggest risk

Now you've identified the biggest risk it's time to commit to doing the hard thing. Running directly into that of which you are most scared. This is never easy, nor do I believe it gets much easier over time. However, as noted above, there is no "success" nor "failure" in this approach. Either you gain conviction or your avoid wasting your time.

“We must all suffer from one of two pains: the pain of discipline or the pain of regret. The difference is discipline weighs ounces while regret weighs tons.” - Jim Rohn

3. Frame your hypothesis in the negative

By this I mean; if your intuition is that SMEs lack an adequate solution for paying international payroll, then your test should be that "SMEs have perfectly adequate solutions for paying international payroll."

This may seem like a trivial distinction, but the importance is to commit yourself to not half-assing the test in hopes you get the result you "want". You should be trying your utmost to prove that your idea, in whole or in part, is unworkable in some way. Only when you are confident* that the hypothesis is wrong can you deem the hypothesis test a "failure" (in which case your initial intuition is correct).

4. Pre-commit to a failure metric

Pushing hard to achieve hypothesis test failure is important, but only if the test is fair to begin with. You should seek to set a clear and testable hypothesis by pre-committing to the condition that would indicate test failure. In the example above you might commit to mom test interviewing 50 SME employees responsible for payroll and deeming the test a failure only if >75% of those interviewed were unhappy with their international payroll provider/process.

In reality it's likely there are many hypothesis tests to be run, including whether there is any consistency in the unhappiness. In each case however, you should be committing ahead of time to a metric that determines pass/fail.

Be as ruthless with your failure metrics as you would be ambitious with your revenue metrics. The worst outcome for startups is not outright failure, it's mediocre success. Don't give yourself false hope by setting the bar too low.

5. Iterate and improve

After you've conducted each test there are 2 possible outcomes - the hypothesis is validated or invalidated, with your initial intuition taking the inverse result.

  • If Invalidated (intuition correct) - what is the new stack ranking? - in the scope of the test have new risks surfaced? Otherwise move onto the next big risk to test.
  • If Validated (intuition incorrect) - great news! You've avoided wasting your time and energy. Has the test given you new ideas for the next avenue to explore? Follow the The Ideation Flywheel.

A note on confidence

The real world is messy - it would be great if guaranteed success was the natural result of ticking off a list of your biggest risks. You will never get that kind of confidence - there will often be unknowables, or indefinite tests. This is where you need to use good judgement and intuition.

A helpful heuristic to jump when you are 80% sure. Less than 80% is wishful thinking and waiting for more is procrastination. It’s a crude metric, if you can even call it a metric at all, but it at least forces a semblance of objectivity.

Failure Modes and Model Suitability

As with any mental model it’s worth asking 2 questions:

  1. What could go wrong if you follow this model to perfection (or follow it blindly)?
  2. What does this model ignore about the world?

As I see it, the risks of being too hypothesis driven are as follows:

  1. Losing sight of the ultimate goal - As mentioned, success in startups is not the absence of risk, it's making something people want and giving it to them in a way that is profitable for your business.
  2. It's easy to procrastinate by testing every risk - don't jump at shadows, some risk is worth taking, particularly if it's reversible (one-way vs. two-way door decisions).
  3. This model is most appropriate for certain types of businesses - there is no probably no good hypothesis test for customer behaviour change, or regulatory change for example. It's hard to know what teens in 2024 (or 2025 or 6) are going to want to do online. You can test and test but at some point you need to defer to your gut.
  4. This model doesn't account for the unknowable risks - you can only test the risks you can think of and there’s a fairly decent chance the biggest risk is something no one on your team could have thought of. What this model does allow you to do is stack rank risks as they present themselves however.