The Hidden History of Evidence-Based Everything (with Helen Pearson)
Helen Pearson, journalist and editor for Nature, discusses how evidence reveals what truly works, highlighting the dangers of untested conventional wisdom and the importance of scientific rigor. She explores the evolution of evidence-based practices in medicine and other fields, and how to navigate a world of complex information.
Deep Dive Analysis
18 Topic Outline
The Danger of Untested Conventional Wisdom
Tragic Case of Sudden Infant Death Syndrome (SIDS)
The Importance of Synthesizing Evidence
Intuition Versus Evidence: When It Fails
Pushback Against Evidence-Based Practices in Medicine
Components of Evidence-Based Decision Making
The Role of Pre-processed Evidence and Clinical Guidelines
Leveraging Systematic Reviews and AI Tools
Understanding the Hierarchy and Limitations of Evidence
The Power and Challenges of Randomization
Ethical Considerations in Randomizing Interventions
The Slippery Nature and Abuse of the Term "Evidence"
Trust in Science and Political Polarization
Impact of AI and Social Media on Evidence
Communicating Evidence to Skeptics
Embedding Evidence Through Training and Accountability
Effective Practices in Evidence-Based Education
Personal Application of Evidence-Based Thinking
7 Key Concepts
Eminence-Based Medicine
A medical practice where decisions are primarily based on the opinions and authority of senior or highly respected doctors, rather than on scientific evidence. This approach was common before the rise of evidence-based medicine.
Randomized Controlled Trial
A study design where similar participants are randomly assigned to either a treatment group or a control group to minimize confounding factors, allowing researchers to more confidently determine if a treatment itself causes an outcome.
Systematic Review
A comprehensive and rigorous method of synthesizing all available research evidence on a specific question, using explicit and transparent methods to identify, select, and critically appraise relevant studies.
Meta-Analysis
A statistical technique often used within a systematic review to combine the quantitative results from multiple independent studies, providing a more precise estimate of an effect than any single study could.
Pre-processed Evidence
Scientific evidence that has already been gathered, synthesized, and distilled into practical formats like clinical guidelines or policy recommendations, making it more accessible and efficient for practitioners to use.
Motivational Interviewing
A communication technique used to address resistance to evidence, which involves asking curious questions, not belittling concerns, offering accurate information, and sometimes sharing personal opinions.
Metacognition
The process of thinking about one's own thinking and learning, where individuals reflect on how they learn best and adjust their strategies accordingly to improve their understanding and retention of information.
10 Questions Answered
Many common practices and beliefs are not rigorously tested and can sometimes backfire with tragic consequences, as seen in the historical advice for infant sleeping positions.
It is crucial to synthesize all available evidence through systematic reviews and meta-analyses rather than relying on single studies, which can be flawed or misleading.
Intuition tends to work well in situations with many examples and tight feedback loops (e.g., boxing, chess), but often fails in complex scenarios like choosing charities or public policies where feedback is delayed or absent.
Resistance often stems from biases such as appeal to authority, commitment effects, and the pain of admitting past decisions or long-held practices were wrong.
It involves considering scientific evidence, the expertise and experiential wisdom of professionals, and the individual values and preferences of the person for whom the decision is being made.
Laypeople can use specialized AI tools like Consensus or Illicit to search for systematic reviews and meta-analyses, which often include plain-language summaries of complex research.
RCTs are powerful because random assignment helps to evenly distribute all other potential influencing factors between groups, allowing researchers to be more confident that any observed difference is due to the treatment being tested.
It's effective to avoid belittling concerns, ask curious questions, offer accurate information, share personal opinions, and openly acknowledge uncertainties or flaws in the evidence.
Medicine benefits from a strong tradition of teaching evidence-based practices in medical schools and a clear system of accountability where practitioners can face consequences for not following evidence-based guidelines.
Practices like metacognition (helping children understand their own learning) and specific, actionable feedback are highly effective, while others like general class size reduction or homework have less strong evidence.
9 Actionable Insights
1. Synthesize All Available Evidence
When evaluating a topic, look beyond single studies and seek out systematic reviews or meta-analyses that combine findings from multiple studies to get a clearer, more reliable picture, as individual studies can be flawed or misleading.
2. Challenge Conventional Wisdom
Actively question common practices or beliefs, especially those based on intuition or anecdote, as they may be unsubstantiated and potentially harmful, as demonstrated by the tragic history of front-sleeping advice for infants.
3. Utilize AI for Evidence Search
Use specialized AI tools like Consensus or Illicit to efficiently search scientific literature and filter for systematic reviews, helping to quickly make sense of large volumes of information while being mindful of potential ‘hallucinations’.
4. Acknowledge Uncertainty Transparently
When discussing scientific findings, be open about uncertainties, limitations, and potential flaws in the evidence, as this approach can build trust and make information more palatable to others.
5. Prioritize Relational Communication
To effectively convey evidence, focus on building trusted relationships and engaging in open conversations, as simply issuing didactic reports or stating facts is often ineffective in getting evidence used.
6. Consider Individual Values in Decisions
Remember that evidence alone doesn’t dictate action; always factor in individual values, preferences, and unique circumstances when making decisions, especially in health or policy, as evidence provides a guide, not a final decision.
7. Advocate for Evidence-Based Training
Support the integration of evidence-based practices into professional training for fields like education, policing, and business, mirroring medicine’s approach to ensure new practitioners are equipped to use scientific findings.
8. Promote Metacognition in Learning
Encourage children to understand their own learning processes by reflecting on what strategies help them grasp concepts, as this self-awareness is a highly effective educational practice for improving understanding and retention.
9. Provide Specific Educational Feedback
In educational settings, offer specific feedback that clearly guides children on how to change their behavior or approach, rather than vague or general comments, to significantly improve learning outcomes.
7 Key Quotes
If we had synthesized the evidence, like at what point would it have been clear that there was that link? And they showed that actually, it would have been apparent after the first two studies in 1970. And they calculated that if that had been done, and the evidence had been synthesized, then approximately 50,000 deaths of children could have been prevented.
Helen Pearson
For doctors being confronted with evidence showing that your practice is wrong is like being hit in the intellectual testicles, because it's just so incredibly painful, you know, to sort of realize and be confronted with evidence, you know, which shows that what you've been doing is wrong.
Helen Pearson
Evidence alone is not enough to tell you what to do.
Helen Pearson
The problem with looking at one study at a time is that, you know, studies can be wrong by chance or they were poorly done... But what we need to be doing is looking at the body of evidence as a whole.
Helen Pearson
Evidence is always changing. It's often uncertain. It can be flawed... It can be biased. Nowadays, it can be, you know, AI contaminated. And it's only ever one factor in a decision.
Helen Pearson
It's not that people are losing trust in science, it's just that scientists are losing influence in this kind of world that we're in, and that that's maybe the problem that needs to be tackled.
Helen Pearson
Building relationships, trusted relationships is a really important part of it, it seems, but it's, there's some quite kind of interesting lessons I think that come out of evidence-based medicine.
Helen Pearson
1 Protocols
Communicating with Evidence Skeptics (Motivational Interviewing)
Helen Pearson- Do not belittle their concerns; be open and respectful.
- Ask curious questions to understand their perspective and underlying reasons.
- Offer accurate information, ideally from a trusted source like a healthcare provider.
- If you don't have all the evidence, offer your personal opinion or experience (e.g., 'I chose to vaccinate my children because I felt the benefits outweigh the risks').
- Acknowledge uncertainties and potential flaws in the evidence to build trust and transparency.