How broken is social science? (with Matt Grossman)

Sep 2, 2021 1h 5m 23 insights Episode Page ↗
Spencer Greenberg and Matt Grossman discuss the evolving quality and reproducibility of social science. Grossman argues it's never been more rigorous, relevant, or self-reflective, despite inherent challenges, highlighting progress in methods, data, and diversity.
Actionable Insights

1. Increase Diversity in Social Science

Actively incorporate more women, racial minorities, and international perspectives into social science research to broaden topics, diversify interpretations, and uncover biases from past research.

2. Include Diverse Political Perspectives

Ensure conservatives remain part of science and engage in the process, commenting on research to mitigate biases that arise from an overwhelmingly liberal social science community and lead to less generous interpretations of political opponents.

3. Self-Reflect on Political Biases

Researchers should critically examine how their own political perspectives, such as being liberal, might be influencing their investigations and interpretations of evidence, especially on politically charged topics.

4. Adopt Different Thinking Habits for Predictions

To improve predictions, individuals should be taught to adopt different thinking habits, as broad knowledge of general social trends can be more effective than deep saturation in one area of expertise.

5. Work in Groups for Better Predictions

Collaborate in groups to make better predictions, as team science and worldwide interactions in interpreting social events can help protect against basic human biases.

6. Narrow Scope of Claims

Be humble and narrow the scope conditions of your claims, avoiding broad pronouncements or reducing complex historical processes to a few variables, which is a productive trend in social science.

7. Leverage Others to Identify Biases

Recognize that while we are not great at understanding our own biases, we are very good at identifying others’ biases, so engage with evidence collected by other people to more closely approximate the truth through a social process.

8. Embrace Unlearning Falsehoods

View the discovery of problems in past studies and the unlearning of previously held beliefs as progress, as this process of re-evaluation regularly helps refine the truth of claims.

9. Be Cautious with Popularized Research

Exercise caution when consuming popularized research, as the journey from scholarship to popularization often involves skipping verification steps, making it more likely to contain unverified claims.

10. Engage in Public Debate on Claims

Foster public debate about big claims, as this process allows for pushback with different kinds of evidence, helping to refine the truth and narrow the application of broad claims.

11. Distinguish Exploratory from Confirmatory Research

Be clear about whether research is exploratory or designed for confirmation of theory, as recognizing this distinction is crucial for evaluating the findings and their implications.

12. Utilize Pre-registration for Studies

Consider adopting pre-registration for studies, where a plan for analyses and data collection is submitted beforehand, as it is a rising intervention that improves rigor despite its challenges.

13. Foster Cross-Field Interaction

Promote more translation and interaction between different fields, such as media, think tanks, academia, and politics, to flatten attempts to understand the world and make it easier for scholars to find and recognize evidence.

14. Inform Public Conversations with Evidence

Make an effort to respond to real-world public conversations and bring the best available evidence to inform those discussions, increasing the relevance of social science.

15. Diversify Evidence & Decision-Makers

Incorporate diverse types of evidence and include different types of people in the decision-making chain when applying research findings, especially for actionable interventions.

16. Acknowledge Historical/Contextual Bias

Be aware that research findings from one time period or culture may not apply to others, as the diversity of human experience means causes and outcomes can differ across time and populations.

17. Utilize Evidence-Based Self-Improvement

Seek self-improvement strategies that are rooted in repeated research results and have been successfully translated into applied forms, such as cognitive behavioral therapy.

18. Engage with Real-World Practitioners

Recognize that innovations often come from the engagement of academic professions with actual actors in the field, rather than solely from within the profession itself.

19. Value Critical Perspectives

Acknowledge and value the perspectives of critics, as their critiques, even if disagreed with, are crucial for stimulating broad efforts and driving the development and improvement of social science.

20. Change Institutions & Share Information to Improve Behavior

Recognize that even when actors have incentives to avoid progress, institutional changes and making people aware of new information can lead to improved behavior and outcomes.

21. Publish Data Sets

Publish data sets alongside research to promote transparency and allow others to replicate findings and build on the science.

22. Implement Pre-publication Data Checking

Utilize pre-publication data checking in journals to ensure the rigor and reproducibility of research before it is published.

23. Learn from Research Practices

Engage in a continuous learning process from research practices, such as pre-registration or data checking, to better understand the nature of research and refine methodologies.