How can we make science more trustworthy? (with Stuart Ritchie)

Jan 19, 2023 1h 26m 14 insights Episode Page ↗
Stuart Ritchie, a Lecturer at King's College London, discusses making science trustworthy, examining controversies like COVID treatments (ivermectin, vitamin D) and the reliability of IQ research. He emphasizes critical thinking and understanding research limitations.
Actionable Insights

1. Be Skeptical of All Claims

Remember that “people make stuff up, even people you like.” Apply critical thinking and skepticism to all information, as deliberate falsehoods or self-deception can occur.

2. Evaluate Scientific Studies Critically

Don’t blindly trust scientific claims, even from top journals. Look for transparency, justified methods, and adequate sample sizes, and be wary of studies lacking these qualities.

3. Prioritize Randomized Controlled Trials

When evaluating treatments or interventions, prioritize evidence from randomized controlled trials (RCTs) as they are the gold standard for establishing causality and are less prone to confounding factors than observational studies.

4. Expect Decline in Effect Sizes

Be aware of the “decline effect,” where initial studies often show larger effects that diminish in subsequent, more rigorous research. This means early findings may overstate the true impact.

5. Avoid Overstating Claims

When advocating for a finding or treatment, avoid making outrageously strong claims, even if you believe in it. Overstating benefits can erode trust and make your position seem less credible.

6. Trace Information to Source

When encountering claims, especially controversial ones, trace them back to their original source to verify their provenance and avoid spreading made-up stories.

7. Question “Controlled For” Claims

When researchers claim to have “controlled for” variables in observational studies, ask for specifics on how they measured and controlled those factors, as measurement error and over-controlling can invalidate conclusions.

8. Map Causal Assumptions

Before drawing conclusions from data, map out the assumed causal relationships between variables using tools like directed acyclic graphs. This practice helps reveal hidden assumptions and encourages circumspection about the data’s true causal structure.

9. Identify Controversial Claims

Seek to identify when a claim is controversial, meaning many people disagree with it, rather than just accepting it as objective fact. This awareness should prompt greater skepticism and a second thought before immediately adopting the claim as true.

10. Utilize Learning Strategies

Employ effective learning strategies from cognitive psychology to learn more efficiently, regardless of your innate intelligence level. Teachers should also equip students with these tools.

11. Cultivate Conscientiousness

Develop high conscientiousness, as it can compensate for lower cognitive abilities by enabling highly organized and productive work habits, leading to success in demanding fields.

12. Focus on Specific Skill Development

Instead of solely focusing on general intelligence, identify specific domains or skills you want to improve and actively practice them. You can get better at any particular domain through effort and dedicated practice.

13. Understand IQ as Learning Potential

View IQ as a measure of potential to pick up and learn new skills faster, rather than an absolute limit. Higher IQ can accelerate skill acquisition, but lower IQ does not prevent learning, it just may take longer.

14. Broaden Social Exposure

Actively seek experiences and interactions with people across the full range of intelligence and educational backgrounds. This can help align your subjective understanding of intelligence with the broader evidence.