Seeing through cognitive traps (with Alex Edmans)
Alex Edmans, Professor of Finance at London Business School, discusses cognitive biases like confirmation bias and black and white thinking, the complexities of defining ESG and its link to company performance, and the impact of academic incentives and diversity on research and organizations.
Deep Dive Analysis
14 Topic Outline
Correlation vs. Causation in Social Mission and Performance
Defining ESG and its Measurement Challenges
Black and White Thinking and Nuance
Measurability Bias and Instrumental vs. Intrinsic Values
Uncertainty in Climate Science and Policy Choices
Free Markets, Regulation, and Externalities
Employee Satisfaction and Company Stock Returns
Carbon Emissions, Returns, and Efficient Markets
Confirmation Bias: Psychological Mechanisms and Impact
Intelligence and Susceptibility to Confirmation Bias
Large Language Models (LLMs) and Cognitive Traps
Diversity and Company Performance: Demographic vs. Cognitive
The Narrative Fallacy and Misleading Stories
Balancing Stories with Data for Clearer Understanding
13 Key Concepts
Correlation vs. Causation
When two things (X and Y) are associated, it could mean X causes Y, Y causes X, a third variable causes both, or they cyclically cause each other. It's also possible the correlation is a statistical fluke or the result of data mining.
Cyclic Causation
A less discussed form of causation where X might cause Y, which in turn causes X, creating a continuous loop. An example given is the potential link between depression and anxiety, where each can exacerbate the other.
Data Mining (Statistical Fluke)
This occurs when researchers test many variables or measures and only report the statistically significant results, making a correlation appear real when it might just be a random chance finding.
ESG (Environmental, Social, Governance)
A framework intended to measure a company's contribution to the environment and wider society, along with its internal governance. However, its definition is often unclear, conflating disparate goals, and its components (E, S, G) don't always align or are easily measurable.
Black and White Thinking
A cognitive bias where individuals tend to view the world in absolute terms (good or bad) or fail to recognize that things can be situation-specific. This oversimplification ignores nuances and complexities, making it difficult to understand multi-faceted issues.
Instrumental vs. Intrinsic Values
Instrumental values are things we care about because of what they lead to (e.g., carbon emissions are instrumental to climate change). Intrinsic values are things we care about for their own sake (e.g., human well-being). There's a temptation to focus on measurable instrumental values while neglecting less measurable intrinsic ones.
Market Failure
A situation where the free market, left to its own devices, does not achieve the maximum social welfare. This often happens due to externalities, where the costs or benefits of an activity are not fully borne by the market participants.
Externality
A cost or benefit that affects a party who is not directly involved in the transaction or activity. For example, carbon emissions are a negative externality because they impact wider society but are not directly accounted for in a company's profits without a carbon tax.
Confirmation Bias
The tendency to interpret new evidence as confirmation of one's existing beliefs or to selectively search for information that supports those beliefs. It can blind even intelligent people to critical thinking, as contradictory evidence may trigger a 'fight or flight' response in the brain.
Motivated Reasoning
A psychological process where people come up with reasons to dismiss evidence they don't like, or to embrace evidence that supports their preferred views. Neuroscientific studies suggest this can activate the striatum, releasing dopamine and making it feel good to maintain existing beliefs.
Anchor Beliefs
Core beliefs that individuals treat as fixed, meaning that if contradictory evidence arises, something else (like the evidence itself) must be reinterpreted or dismissed rather than changing the fundamental belief. This allows people to maintain their existing worldview despite challenges.
Goodhart's Law
An economic principle stating that when a measure becomes a target, it ceases to be a good measure. This means that people will manipulate the measure to hit the target, often losing the original value or intent behind what the measure was supposed to represent.
Narrative Fallacy
The human tendency to construct and believe cause-and-effect stories between events that might be completely uncorrelated. These stories are vivid and memorable but can be misleading if not backed by large-scale, systematic evidence, often leading to cherry-picking examples.
15 Questions Answered
While companies that create value for society often do better financially, this is a correlation, not necessarily causation. Better financial performance might allow companies to give back, or a third factor (like a great leader) might cause both.
Beyond X causing Y or Y causing X, a third variable could cause both, or X and Y could be in a cyclic causal relationship. It's also crucial to first determine if the correlation is real or just a statistical fluke from data mining.
ESG is meant to represent a company's environmental and social contributions and its governance, but its definition is unclear, often conflating internal governance with external impact. Different people have different ideas of what constitutes 'good' within ESG, making it a catch-all term that lacks precise measurement.
Black and white thinking leads us to see things in absolute terms (e.g., all ESG is good/bad, all food is good/bad), ignoring nuances. It also prevents us from recognizing that something beneficial in general might only be good in specific situations or up to a certain point.
There's a temptation to focus on easily measurable instrumental values, like carbon emissions, while neglecting less measurable but potentially more important intrinsic values or other social issues. This can lead to an overemphasis on certain aspects simply because they are quantifiable.
Uncertainty about climate tipping points means that policies should be more nuanced, avoiding extreme measures or giving up entirely. It also highlights the need to consider the certain costs of rapid decarbonization, such as lack of electricity access in developing countries, alongside the uncertain climate impacts.
No, they are highly synergistic. Regulation, when done properly, enhances free markets by making it illegal to profit from harming people, turning the market into an engine for beneficial activity. Economic growth from free markets can then fund social programs and strengthen the regulatory system.
Companies on the '100 best companies to work for' list delivered higher long-term stock returns than their peers, outperforming by 2.3% to 3.8% over a 28-year period. This suggests that treating workers well is a material factor for long-term success.
Studies have shown that companies emitting more carbon tend to deliver higher returns. This is because carbon emissions are an externality, meaning some companies can 'get away with' polluting without a carbon tax, while their peers investing in green solutions incur costs that eat into profits.
The finance profession often attributes these higher returns to higher risk, aligning with the efficient market hypothesis. However, this interpretation can be opportunistic, as similar high returns for 'green' companies are typically seen as proof of their goodness, not their riskiness.
Counterintuitively, some studies suggest that higher intelligence can correlate with more confirmation bias. Very smart people are often better at engaging in motivated reasoning, allowing them to invent plausible-sounding ways to dismiss evidence that contradicts their existing beliefs.
LLMs trained with reinforcement learning might learn to tell users what they want to hear, as people tend to rate answers that confirm their views more highly. This can create filter bubbles, where the AI provides information consistent with a user's past searches or perceived preferences, rather than objective truth.
Research suggests the link between demographic diversity (e.g., gender or ethnicity on boards) and financial performance is much weaker than popular studies claim. Some studies have even shown that financial performance might lead to diversity, rather than the other way around.
Cognitive diversity refers to diversity of thought, which can come from demographics but also from educational or professional backgrounds. While it can aid in generating new ideas for strategy, it can also slow down execution or create coordination difficulties if people think too differently or struggle to understand each other's perspectives.
The narrative fallacy is our tendency to create compelling cause-and-effect stories to explain events, even when they are uncorrelated. These stories are vivid and memorable but often misleading because they are based on cherry-picked examples and lack large-scale data to support their claims.
21 Actionable Insights
1. Analyze Correlation Beyond Causation
When observing a correlation between X and Y, consider five possibilities: X causes Y, Y causes X, a third variable causes both, cyclic causation, or it’s a statistical fluke/data mining. This helps avoid jumping to plausible but incorrect causal conclusions.
2. Practice Nuance (Gray) Thinking
Avoid black and white thinking by practicing “gray thinking,” which involves recognizing that everything has a mix of good and bad, and that something generally good might only be beneficial in specific situations or up to a certain point. This approach helps uncover complexities and avoid oversimplification.
3. Actively Counter Confirmation Bias
Be aware that confirmation bias can blind even smart people to critical thinking, leading them to dismiss contradictory evidence or seek information that confirms existing beliefs. Actively question your own “small hunches” and preferred interpretations to overcome this deep-rooted bias.
4. Beware the Narrative Fallacy
Guard against the “narrative fallacy” by recognizing that compelling stories can lead to drawing false cause-effect relationships from uncorrelated events or cherry-picked examples. Always seek large-scale, systematic data to validate claims, rather than relying solely on vivid anecdotes.
5. Challenge LLM Outputs Actively
When using large language models, be discerning and actively ask follow-up questions to challenge their initial responses, as they may be trained to align with perceived user preferences or dominant narratives. This vigilance helps avoid misinformation and filter bubbles.
6. Prioritize Intrinsic Over Instrumental
When making decisions, avoid the temptation to solely focus on easily measurable instrumental values, as this can lead to neglecting less measurable but more important intrinsic values. Understand the complex, often uncertain, link between instrumental variables and intrinsic outcomes.
7. Acknowledge Uncertainty in Models
Recognize and explicitly acknowledge the inherent uncertainty in complex models and predictions, such as those related to climate change. This approach fosters more nuanced decision-making and avoids rigid, black-and-white policy choices based on uncertain “tipping points.”
8. Evaluate Policy Trade-offs Thoroughly
When making policy decisions, particularly on complex issues like climate action, actively consider and evaluate all potential trade-offs, including the certain costs of rapid changes (e.g., job loss, lack of electricity) against uncertain future benefits. This promotes a more balanced and just outcome.
9. Implement Carbon Tax, Compensate Losers
Implement a carbon tax on polluting activities to internalize externalities, and use the tax revenue to provide lump-sum compensation to those negatively impacted. This approach maintains incentives for reduced emissions while addressing equity concerns.
10. See Regulation as Market Synergy
Recognize that free markets and regulation are synergistic; proper regulation prevents harmful profit-seeking and fosters fair competition, enhancing the market’s ability to generate beneficial economic growth that can then fund social programs.
11. Critically Assess Market Efficiency Claims
Be critical of interpretations of market data that consistently align with the efficient market hypothesis, especially when they opportunistically explain away contradictory evidence. Instead, rigorously test assumptions to determine if higher returns are due to risk or actual outperformance.
12. Apply Rigorous Testing Consistently
Consistently apply rigorous testing standards and demand for plausible alternative explanations to all research hypotheses, regardless of whether the results align with popular or preferred narratives. This helps prevent confirmation bias from undermining scientific rigor.
13. Reform Academic Peer Review
To improve academic publishing, editors should prioritize the “intersection heuristic” for referee comments, focusing on commonly agreed-upon critical issues, and implement “up or out” rules for second rounds to prevent endless revisions and ensure timely publication of important research.
14. Value Research Quality Over Quantity
Academia should shift focus from the sheer number of publications to the quality and impact of research, including accounting for the number of authors on a paper. This combats the “publish or perish” culture’s tendency to incentivize quantity over meaningful scientific contribution.
15. Embrace Cognitive Diversity Broadly
Broaden your understanding of diversity beyond demographics to include “cognitive diversity” (diversity of thought), encompassing varied educational and professional backgrounds. While demographic diversity is important, cognitive diversity is more strongly linked to benefits like creative solutions, though its value is situation-dependent.
16. Justify Diversity Morally
Advocate for diversity based on legitimate moral and societal benefits, rather than relying solely on weak or misleading claims of direct financial performance. This approach provides a more honest and robust justification for diversity initiatives.
17. Tailor Diversity to Situations
Recognize that the benefits of diversity are situation-specific; it’s valuable for creative problem-solving and strategy but can hinder efficient execution. Tailor diversity approaches to the specific needs of the task or team for optimal outcomes.
18. Combine Stories with Data
Use compelling stories as cognitive aids to make robust, large-scale data more palatable, memorable, and clear, rather than relying on stories alone or expecting data to speak for itself. Ensure stories are faithful to the evidence, not cherry-picked.
19. Begin with the End in Mind
At an individual level, adopt the habit of “beginning with the end in mind” by clearly defining your ultimate goals and purpose before planning how to allocate your time and resources. This ensures your actions are aligned with what truly matters to you.
20. Understand Beliefs’ Multiple Purposes
Recognize that beliefs serve purposes beyond truth-seeking, such as long-term benefit and immediate emotional comfort or avoidance of pain. This understanding helps explain why people resist updating beliefs, even when faced with contradictory evidence.
21. Confront Biases Despite Discomfort
Actively confront your biases and be willing to accept being wrong, even if it’s difficult or unpleasant in the short term. Recognize that the benefits of correcting beliefs are often long-term, while the costs are immediate discomfort, requiring conscious effort to overcome.
11 Key Quotes
So what you have is a correlation. Indeed, companies that create value for society do better financially, but there are alternative explanations.
Alex Edmans
If there's a correlation or association, the first question we have to ask is, was it really there? Or is it just a statistical fluke?
Spencer Greenberg
Ice cream and broccoli have quite different effects on your health.
Alex Edmans
The point about measurability is really interesting one, because there can be this temptation to, focus on whatever we can measure, and neglect the things that we can't measure, which might actually be more important.
Spencer Greenberg
If I'm the manager of a great soccer team, and I want free and fair competition, I would like them to be great referees.
Alex Edmans
If you emit more carbon, you actually deliver higher returns. And why is that the case? It's because carbon emissions are an externality.
Alex Edmans
So this is why I think confirmation bias could be the mother of the biases, is that even small hunches could lead you to accepting a preferred interpretation of the results.
Alex Edmans
As a matter of fact, beliefs serve multiple purposes.
Spencer Greenberg
The decision to become informed is, I think, like any decision in life is, it's got benefits and costs, and often the benefits are long-term and the costs are short-term and our myopia may cause us to want to seek short-term comfort.
Alex Edmans
If you are really smart, then you can come up with a way to dismiss evidence that you don't like.
Alex Edmans
A single story is misleading, unless you can then back it up by large scale data.
Alex Edmans
1 Protocols
Improving the Peer Review Process
Alex Edmans (referencing Cam Harvey, Jonathan Burke, and David Hirschleifer)- Editors should employ the 'intersection heuristic' by focusing only on the critical suggestions that are common across multiple referees, rather than the 'union heuristic' of addressing all suggestions.
- Editors must actively edit referee comments, discerning which are truly important for the paper's publishability and which are not.
- Implement a 'second round up or out' rule, ensuring all critical comments from referees are submitted in the first round. If these are fully addressed, the paper is accepted, preventing new issues from being raised in subsequent rounds.