Measuring everything that matters (with Doug Hubbard)

May 22, 2024 1h 12m 18 insights Episode Page ↗
Spencer Greenberg speaks with Doug Hubbard about quantitative methods, biases in risk tolerance, and techniques for better prediction, emphasizing that everything that matters can be measured indirectly through its observable consequences.
Actionable Insights

1. Doubt Your Gut Feel

Actively doubt your intuition and gut feelings when making consequential decisions, as quantitative assistance consistently leads to better outcomes than relying solely on subjective judgment.

2. Measure Decision-Making Skill

Regularly measure your own skill at decision-making to identify conditions and methods that improve your choices, as this is a meta-measurement that informs all other decisions.

3. Calibrate Probability Assessments

Improve your probabilistic prediction accuracy by undergoing calibration training, which involves making numerous estimates, receiving immediate feedback on their correctness, and iteratively refining your confidence levels.

4. Track Decision Performance

Systematically document and track your predictions and decisions, then compare them against actual outcomes to receive concrete feedback on your performance and improve decision-making over time.

5. Prioritize Skepticism

Cultivate personal skepticism about your own beliefs and confidence, asking ‘Why do I think I’m as good as I am?’ to avoid self-delusion and foster improvement.

6. Quantify Uncertainty with Probabilities

For important decisions, explicitly quantify your uncertainty by thinking in terms of probabilities, as this allows you to make better ‘bets’ and update your understanding based on new observations.

7. Calculate Value of Information

For significant decisions, compute the ‘value of information’ for each uncertain variable by multiplying the chance of being wrong by the cost of being wrong, to identify which measurements are most economically valuable to pursue.

8. Reverse Measurement Inversion

Counter the ‘measurement inversion’ by consciously prioritizing the measurement of highly uncertain variables that have significant impact, rather than focusing on easily measurable but less impactful factors.

9. Measure High Uncertainty First

Focus measurement efforts on variables with the highest uncertainty, as even a few observations in these areas can significantly reduce overall uncertainty more efficiently than measuring already well-known factors.

10. Quantify Personal Decisions

For significant personal decisions like home renovations, starting a business, or major purchases, conduct quantitative analysis, forecasting, and data gathering to reduce failure rates and improve outcomes.

11. Identify Observable Consequences

When trying to measure something seemingly abstract, identify its observable consequences; this helps define what you mean by it and is the first step towards measurement.

12. Apply Equivalent Bet Method

When assessing probabilities, use the ’equivalent bet’ method by comparing your confidence in a claim to a chance-based gamble; if you prefer the gamble, your stated confidence is likely too high.

13. Utilize Klein’s Pre-Mortem

To identify potential risks and failures, use Klein’s pre-mortem technique by imagining the project has already failed in the future and then explaining what went wrong, which encourages more candid risk assessment.

14. Assume Your Answer’s Wrong

After forming an answer or estimate, deliberately assume it is wrong and articulate reasons why, which can help uncover overlooked factors and lead to a more accurate final assessment.

15. Start Wide, Chip Tails

When creating confidence intervals for continuous values, begin with an extremely wide range you are highly certain contains the true answer, then progressively narrow it by using the equivalent bet method to remove the least likely extremes.

16. Assess Partner Financial Viability

Evaluate a potential long-term partner’s financial responsibility, such as their credit score and attitudes towards saving, to prevent future problems when mixing assets in marriage.

17. Apply the Rule of Five

To quickly estimate the median of any large population, randomly select five samples; there is a 93.75% probability that the true population median lies between the minimum and maximum values of your five-item sample.

18. Verify Method Effectiveness

Actively question whether your decision-making methods actually improve outcomes, rather than merely increasing confidence, and seek evidence-based approaches that are proven to be effective.