Are the culture wars harming science? (with Cremieux)
In this episode, Spencer Greenberg speaks with Cremieux about how leveraging large datasets can simplify complex problems and resolve misinformation. They discuss critical evaluation of research, the impact of culture wars on science, and the societal implications of heritability and IQ.
Deep Dive Analysis
14 Topic Outline
Data Gathering to Solve Complex Problems
Fraud and Poor Methodology in Research
Applying Data-Driven Solutions Beyond Medicine
Challenges of Solving Social Problems vs. Biological Problems
Balancing Power Between Companies, Unions, and Consumers
Japanese and Swedish Models for Technology Adoption in Labor
Misinterpretation and Misinformation in Research
Heuristics for Evaluating Academic Papers
Impact of Culture Wars on Scientific Research and Data Access
The 'Frontlash' Phenomenon and Public Reaction to Research
Heritability: Twin Studies vs. DNA-Based Methods
The Value and Societal Implications of Knowing Heritability
IQ as a Predictor of Life Outcomes and Skill Acquisition
The Flynn Effect and the Value of Knowing Your Own IQ
6 Key Concepts
P-value
A p-value indicates the likelihood of obtaining a result at least as extreme as the observed one, assuming there is no actual effect. In many social sciences, a p-value below 0.05 is often used as an arbitrary cutoff for statistical significance, leading to incentives for researchers to manipulate data to achieve this threshold.
P-hacking
This refers to the practice of manipulating data analysis or collection until a statistically significant p-value (often below 0.05) is obtained. It can lead to unreliable results, as studies with p-values clustered just below the significance threshold often indicate that authors were trying to make their findings appear significant.
Assortative Mating
This occurs when individuals with similar traits (e.g., high education) tend to marry each other. This phenomenon can complicate heritability estimates in twin studies by increasing the correlation of genes related to those traits in offspring, making it difficult to accurately determine genetic influence.
Frontlash
A phenomenon where, in response to an expected negative reaction or backlash to a contentious event or research finding, people's opinions or actions actually become more positive or helpful. This can lead to better outcomes than initially feared, as individuals may preemptively act to counter anticipated prejudice or harm.
Missing Heritability Problem
This describes the discrepancy between heritability estimates from twin studies (which tend to be high) and those from DNA-based methods (which are typically lower). The gap is thought to be due to the inability of current DNA methods to capture all genetic influences, as well as challenges in measuring complex traits accurately at scale.
Flynn Effect
The observed phenomenon of rising IQ scores over several decades. However, this increase is primarily attributed to people becoming more familiar with test formats and content due to universal education, rather than a genuine increase in general intelligence or cognitive capability.
9 Questions Answered
Many complex problems, especially in fields like drug discovery and archaeology, can be significantly advanced or even resolved by gathering sufficient data and analyzing it rigorously. However, social problems are often harder to fix than biological ones due to human investment in beliefs and resistance to change.
Social problems are more difficult because people are heavily invested in their desired outcomes, often lack a clear causal understanding, and may actively impede progress due to vested interests or resistance to policies that might harm certain groups, even if beneficial overall.
Different models exist: in Japan, firms maintain employment and retrain workers, fostering technology adoption. In Sweden, unions aim to ensure high employment and wages despite new technologies, though the actual impact on employment outcomes compared to the US is not significantly different.
Look for 'p-hacking' indicators, such as many p-values clustered between 0.05 and 0.01. Consider sample size in context: small samples (e.g., 20) are generally unreliable for broad inferences, but can be acceptable for pilot studies or when studying an entire small population. Also, check for pre-registration deviations and whether the conclusions align with the actual data presented.
Culture wars significantly harm science by making it difficult to publish certain conclusions, even if correct, and by restricting access to data for contentious topics. This polarization can lead to a decline in public trust in scientific institutions and impede progress.
Generally, there should be no limits on research topics. While there's a fear of backlash and misinterpretation, studies often show that public reactions are more reasonable and call for more research rather than banning it. Suppressing information can hinder the development of better solutions.
Knowing heritability is crucial for breeding programs (e.g., cattle, embryo selection) and for calibrating expectations about trait changes. It helps in understanding the causes of societal issues and can inform drug discovery or interventions for conditions like schizophrenia.
IQ measures a general aptitude that is important and predicts many aspects of everyday life and societal structure. While not the only predictor (e.g., conscientiousness is also vital), it can be a limiting factor, as lower IQs can make it harder to excel in cognitively demanding fields, even with significant effort.
Generally, no. The obsession with knowing one's own IQ score is often seen as a distraction or a form of self-aggrandizement, rather than a serious pursuit of understanding. It can also contribute to a 'quasi-IQ science-y view' that is not supported by data.
22 Actionable Insights
1. Form Opinions on Harmful Evidence
When presented with substantial evidence of significant harm, recognize that refusing to form an opinion can be a moral failing, as it may enable harmful behavior by preventing necessary action.
2. Leverage Large Datasets for Problem Solving
To resolve complex problems, actively seek and utilize sufficiently large datasets, as this approach has proven effective in areas like drug discovery and debunking medical myths.
3. Critically Evaluate Research Papers
Do not blindly trust scientific claims; instead, critically read the original papers to identify methodological flaws, misinterpretations of data, or comparisons of unlike things, as these issues can invalidate conclusions.
4. Reverse “Just-So” Explanations
To combat confirmation bias, practice reversing your initial explanations for observed phenomena by attempting to construct equally plausible arguments for the opposite outcome or for no relationship at all.
5. Test Obvious Assumptions
Challenge commonly held beliefs and “obvious” practices with empirical testing, as intuition can be misleading, and empirical evidence may reveal the opposite of what is assumed.
6. Combat Research Fraud
Actively scrutinize research for fraud and poor methodology, as these issues can severely impede progress and lead to incorrect conclusions, as seen in the Parkinson’s/Alzheimer’s research setback.
7. Acknowledge Social Problem Complexity
Understand that social problems are inherently more complex than biological ones because individuals are deeply invested in their beliefs and often lack accurate causal understanding, making solutions harder to implement.
8. Advocate for Unrestricted Research
Support the principle of unrestricted scientific inquiry, believing that open research, even on controversial subjects, ultimately leads to a more robust understanding and the production of higher-quality evidence.
9. Recognize Culture War’s Science Impact
Understand that cultural conflicts can obstruct scientific research by making it challenging to publish “negative” or contentious findings and by restricting access to crucial data, ultimately hindering objective understanding.
10. Streamline Data Access
Push for reduced bureaucracy and easier access to large population datasets, as current credentialing and form-filling processes can delay crucial research findings by over a year.
11. Counter Interest Group Obstruction
When facing societal problems, analyze and address the specific motivations of powerful interest groups that may be actively impeding progress, as their resistance can prevent beneficial changes like renewable energy development or port automation.
12. Embrace Technology’s Cultural Impact
Understand that new technologies can fundamentally alter societal norms and reduce stigma around previously controversial topics, as people become accustomed to their undeniable existence and benefits.
13. Ensure Worker Retraining for Automation
To facilitate the adoption of new technologies like automation, companies should commit to retraining employees for new roles, as this approach, seen in Japan, reduces resistance and fosters technological progress.
14. Contextualize Sample Size
When assessing research, consider the sample size in relation to the research question; large samples are vital for generalizable findings, while small samples may suffice for pilot studies or when studying an entire population.
15. Identify P-Hacking in Studies
When reviewing studies, look for a concentration of p-values between 0.05 and 0.01 for key findings, as this often indicates researchers manipulated data to achieve statistical significance.
16. Anticipate Frontlash, Not Backlash
Recognize that people often overestimate the negative reactions (backlash) to controversial scientific findings, when in reality, responses are frequently more constructive, leading to calls for further research.
17. Utilize Heritability for Interventions
Recognize that accurate knowledge of trait heritability can inform critical decisions in areas like selective breeding, embryo selection, and developing targeted interventions for conditions like schizophrenia.
18. Trust Twin Studies for Heritability
For understanding the heritability of traits, rely more on twin studies, as they are a more powerful method that captures all genetic influence, unlike SNP-based methods which are often limited by data quality and sample size.
19. Recognize IQ as a Limiter
Understand that while effort and training are crucial, IQ can act as a limiting factor in achieving high levels of success in certain complex domains, making some achievements less likely for individuals with very low IQs.
20. Combine Training with Aptitude
While dedicated training can improve nearly any skill, understand that inherent aptitude, such as IQ, can influence the ultimate potential and speed of learning, especially in complex tasks.
21. Cultivate Specific Interests
Understand that individual interests play a significant role in developing specific abilities and skills beyond general cognitive aptitude, encouraging people to pursue what genuinely engages them.
22. Disregard Personal IQ Scores
Refrain from fixating on your own or others’ IQ scores, as this obsession is often a distraction from genuine intellectual curiosity and a deeper understanding of complex topics.
5 Key Quotes
The really hard problem of finding drugs for Alzheimer's and Parkinson's might just be the really hard problem of not gathering enough data to use in a more appropriate way and having too many frauds, you know, running about mucking up the ecosystem of thought on the matter.
Cremieux
It seems obvious, but it's just, it just isn't.
Cremieux
I think that there's this idea that you either don't trust studies, like studies are bullshit, or you trust studies, and then, you know, pro-science. But actually, I think both of those ideas are really unhelpful. Rather, you have to think about some papers being reliable and some being unreliable. It's really a spectrum.
Spencer Greenberg
Technology has a way of reforging our cultural intuitions about things.
Cremieux
You can do anything if you have a high IQ, but you can't do everything if you have a very low IQ.
Cremieux