Avoid flopping due to untested assumptions
“Assume nothing, question everything.” - James Patterson
A hypothesis can be
For a hypothesis to be a scientific hypothesis, the scientific method requires that one can test it.
An hypothesis driven approach involves learning and problem-solving by using the scientific method for investigating phenomena, acquiring new knowledge, and correcting and integrating previous knowledge back into our thinking.
Taking an hypothesis driven approach to a problem means attempting to solve that problem by focusing on your best hypothesis as to the answer.
A hypothesis-driven approach is one where you state your assumptions about what you think the answer is, and then fact-find to validate or refute.
The hypothesis driven approach is both dynamic and efficient and means that you will always be moving forward towards a solution to the main problem. You will maintain tight focus and avoiding getting side-tracked trying to do everything.
The hypothesis driven approach is characterised by a constant focus on the main question.
Practicing Hypothesis-Driven Development is thinking about the development of new ideas, products, and services – even organizational change – as a series of experiments to determine whether an expected outcome will be achieved. The process is iterated upon until a desirable outcome is obtained or the idea is determined to be not viable.
We do not do projects anymore, only experiments. Customer discovery and Lean Startup strategies are designed to test assumptions about customers. Quality Assurance is testing system behavior against defined specifications. The experimental principle also applies in Test-Driven Development – we write the test first, then use the test to validate that our code is correct, and succeed if the code passes the test. Ultimately, product or service development is a process to test a hypothesis about system behavior in the environment or market it is developed for.
https://www.alexandercowan.com/hypothesis-driven-development-practitioners-guide/
Whereas bottoms-up problem solving (a non-hypothesis-driven approach) analyzes the data/information to arrive at your answer, top-down identifies an answer and looks to data/information to validate it.
Comparatively speaking, bottoms-up problem solving can be a never-ending process, whereas top-down is laser-focused - and for that reason, effective.
Bottom-up problem-solving begins with understanding the problem statement. Then, one gathers a relevant data set and analyzes it for an indication of the underlying issue. Perhaps this analysis involves graphing the data and performing some calculations. Most often one forms insights and the cycle repeats with further data collection and analysis. In this way, one builds a picture of the problem and solutions over time (thus, bottoms-up).
While the top-down methodology also begins with a clear problem statement, the two paths diverge from there. Top-down problem-solving first attempts to create a MECE model of the problem space. A common way of creating the model is to use an issue tree. Next, one applies first-principles thinking to the problem space to form early hypotheses. These hypotheses are the foundation of the top-down approach and the true 'secret sauce' of its effectiveness.
With an hypothesis approach, there is a need to generate, organize, analyze and communicate the following pieces of information, and the relationship between them:
From Assumption to Hypothesis done right
An assumption is a statement that you’re going to suppose is true (without necessarily attempting to prove), whereas a hypothesis is a statement that you’re actually attempting to prove as opposed to merely assuming.
A hypothesis is a more on point version of an assumption and it is formulated in such a way that it can be tested and its outcome(s) can be easily measured.
| Dimension | Assumption | Hypothesis |
| Cognitive reasoning | Belief | Prediction |
| Rationalization | Axiomatic (self-evident or unquestionable) | Educated guess |
| Supported by | Speculation/no evidence | Reasoning |
| Measurability | Unmeasurable | Measurable |
| Verifiability | Doesn't require any verification | Needs to be validated/rejected |
| Specificity | Vague | Specific |
| Belief in truthiness | Optimistic | Agnostic |
| Testability | Untestable | Testable |
According to Board of Innovation, the different types of assumptions are:
The concept of the "riskiest assumption" in the context of assumption mapping, which is often used in product development and business strategy, refers to identifying and testing the assumption that poses the greatest risk to the success of a project or initiative.
It's the assumption that will make or break your business/product.
This approach is grounded in the lean startup methodology and is vital for efficient resource allocation and minimizing risks.
Here's a breakdown of the concept:
Identification of Assumptions: Initially, a team or individual identifies all the assumptions underlying their project or business model. These assumptions can range from customer behavior and market demand to technical feasibility and cost estimates.
Prioritization of Assumptions: Once assumptions are listed, they are prioritized based on two key factors: the level of uncertainty and the potential impact on the project. The level of uncertainty refers to how much is known about the assumption, while the impact considers how critical the assumption is to the project's success.
Riskiest Assumption: The riskiest assumption is typically one that has both a high level of uncertainty and a high potential impact. This is the assumption that, if incorrect, could most significantly derail or hinder the project. It represents the greatest potential risk.
Testing and Validation: The next step is to devise experiments or tests to validate or invalidate the riskiest assumption as quickly and cost-effectively as possible. This could involve creating a minimum viable product (MVP), conducting market research, or running a pilot program.
Iterative Process: Assumption mapping and testing is an iterative process. Once the riskiest assumption is tested, the team revisits their assumptions, reevaluates their priorities, and may identify a new riskiest assumption to test. This cycle continues, helping to reduce uncertainty and refine the project or business model.
By focusing on the riskiest assumption first, teams can avoid investing heavily in projects with fundamental flaws or misaligned with market realities. It's a strategy that promotes agility and adaptability, key factors in the rapidly changing business and technology landscapes.
Things that that may be considered surprising or counterintuitive.
Having an assumption ideation structure helps to come up with and organize different types of assumptions.
A high level structure for assumptions are:
In combination with this high level structure, other sub structures can be used for ideation, for instance:
The assumption context is the context in which we are ideating assumptions.
For instance, it might be for a new
...we plan to provide, or for a
...of someone we want to help.
Assumptions can be anywhere from highly generic, to super-specific.
An example of a generic behavioural assumption is
"We belive people will use our product".
An example of a specific behavioural assumption is
"We belive people with role/responsibility X will use feature Y x times each day/week/month."
There might be multiple ways of articulating an hypothesis. Here are some examples:
"We believe that [what must be true for a solution to work]. To test this we will [actions the team will take to test hypothesis]. If [described outcome of action / or change in key metric], we will have confidence that our hypothesis was correct."
I believe this will move X objective in Y measurable way.
We believe <this capability>
What functionality we will develop to test our hypothesis? By defining a ‘test’ capability of the product or service that we are attempting to build, we identify the functionality and hypothesis we want to test.
Will result in <this outcome>
What is the expected outcome of our experiment? What is the specific result we expect to achieve by building the ‘test’ capability?
We will have confidence to proceed when <we see a measurable signal>
What signals will indicate that the capability we have built is effective? What key metrics (qualitative or quantitative) we will measure to provide evidence that our experiment has succeeded and give us enough confidence to move to the next stage.
If [assumption] then [condition], because [prediction].
Example:
If I replace the “learn more” CTA on my page with a “request a demo” button, then the number of clicks will increase by 10% in one month because it will make it clear for my audience what the outcome of clicking on the CTA is.
Changing [problem] into [solution] will [anticipated result].
Example:
Changing the number of form fields from 10 to 30 will increase the number of leads by 20% in one month.
A high quality hypothesis is
This may seem like an obvious step, but it's important to be clear about what you're trying to solve.
What have we learned to far that impacts how we perceive this problem?
An initial hypothesis is a proposed explanation for an event or phenomenon that can be tested. It's important to note that a good hypothesis doesn't have to be correct – it just has to be plausible.
For example, let's say you're trying to improve customer satisfaction at your company. Your hypothesis could be that providing more customer service training will improve satisfaction, or perhaps hiring more seasoned employees/agents. This answer-driven approach gets you thinking early about the solution early on.
Once you have evaluated your hypotheses based on these criteria, you can prioritize them by ranking them in order of importance or likelihood of success.
It's important to remember that testing multiple hypotheses in parallel can be a good strategy, as it can help you quickly identify the most promising ones and avoid wasting time on less relevant or impactful ideas.
Dot Vote method
Once you’ve got a list of questions, it’s time to decide which are potentially more impactful for the product. The Dot Vote method, where team members are given dots to place on the questions, helps prioritize the questions and assumptions.
Our team uses this method when we’re faced with many ideas and need to eliminate some of them. We started by grouping similar ideas and use 3-5 dots to vote. At the end of the process, we’ll have the preliminary data on the possible impact and our team’s interest in developing certain features.
This method allows us to prioritize the statements derived from the HMW technique and we’re only converting the top ones.
From there, you're going to flesh out your logic taking a decision tree approach. That's thinking through what needs to be true for your hypothesis to be true. Fast forward and you will end up with a decision-tree with the 1st level being your hypothesis, the 2nd level being your supporting assumptions or logic and the 3rd level downward being the fact points that you'll need to uncover. This tree informs your work plan. Want a real world hypothesis tree to work from? Sign up to the right to claim your free hypothesis problem solving template.
State the indicators to evaluate if the experiment has succeeded.
It is important to seek feedback and validation from multiple sources and perspectives.
Gather the data required to validate or refute your hypothesis. This can be done in a number of ways, including surveys, interviews, focus groups, and data analysis.
Forming very specific hypotheses enables you to quickly determine the data you need to test your hypotheses. It also allows you to only collect the data you need to test your hypotheses.
In the context of the scientific method, we may want to collect data up until the point of ~100% certainty.
In a business setting, we are usually looking for 80/20 answers. That means we will only gather and analyze data to the extent needed to be ~80% certain of the answer. This involves prioritizing speed over 100% certainty.
When we work in an hypothesis driven way, if the cause of some problem is A, B, C or D, we start by assuming the most ostensibly likely option - say A - is the cause and then checking whether this is true or not. If it is not, then we iterate the same process with the other options - B, C and D - in order of their likelihood until we find the correct answer.
This seems pretty sensible, but it is actually very different to how most people will work in practice. Especially if you are coming from an academic or research background, you will likely try to analyse the entirety of a system before going on to focus on any of its parts. This is all very rigorous and appropriate in its own context, but consultants are employed to generate solutions. Analysis is only a means to that end.
A lot of the time, we are looking to understand, predict or adjust behaviour.
Both marketing, sales, product and leadership is a behavioural adjustment game. Basically we are trying to get people to adopt a new type of behaviour, or by change how they are already doing something.
When designing behavioural hypotheses, it makes sense to be aware of the behavioural drivers and behavioural hurdles of the people whose behaviour we are attempting to impact.