What should the Effective Altruism movement learn from the SBF / FTX scandal? (with Will MacAskill)

Apr 15, 2024 2h 1m 16 insights Episode Page ↗
Will McCaskill discusses the FTX collapse, its implications for Effective Altruism, and his new research on post-AGI governance. He shares personal lessons on trust and oversight, and community-wide insights on governance, moral philosophy, and the future of EA.
Actionable Insights

1. Emphasize Strong Governance Systems

Place significant weight on establishing robust governance, including oversight, feedback mechanisms, and incentives, to reduce the incidence of bad behavior in organizations. This is crucial because even successful, admired individuals can commit fraud when governance is poor.

2. Reject “EA Exceptionalism” Mindset

Assume that individuals involved in effective altruism have the same average moral traits as the general population, unless there is strong evidence to the contrary. This helps prevent over-trusting individuals based solely on their affiliation with EA.

3. Embrace Moral Uncertainty, Humility

Avoid being overly certain in any single moral theory, especially utilitarianism, and recognize that complex calculations that violate common-sense morality are likely mistaken. Instead, prioritize being a good citizen and following strong moral rules and heuristics.

4. Question In-Group Trust

Be less apt to trust people, even those perceived as “on the same team” or within the same community (e.g., Effective Altruism), and actively entertain the possibility of bad behavior or deception. This requires personal vigilance against blind trust.

5. Anticipate Worst-Case Outcomes

Pay much more attention to wider error bars and potential worst-case outcomes, such as massive fraud or illicit activities, even if they seem unlikely. This involves proactively considering extreme negative possibilities in planning and assessment.

6. Know Fraud’s High Base Rate

Recognize that the base rate of fraud, even among successful companies and philanthropic individuals, is significantly higher than commonly perceived. This awareness should inform a more cautious and vigilant approach to organizational and personal interactions.

7. Streamline Roles & Responsibilities

Avoid taking on too many different roles or having multiple overlapping responsibilities, as this can lead to burnout and hinder effective action during crises. Instead, aim for greater focus and clarity in one’s commitments.

8. Distrust VC Fraud Incentives

Be cautious of trusting venture capitalists to vet companies for fraud, as their incentives are primarily aligned with potential financial returns and not necessarily with preventing fraudulent behavior. Their focus is on profit, not ethical conduct.

9. Communicate EA’s Broader Morality

Discuss Effective Altruism in a way that clearly integrates virtues like cooperativeness, integrity, honesty, and humility, alongside the distinctive focus on beneficence and truth-seeking. This ensures a more holistic understanding of EA’s values.

10. Decentralize Organizational Leadership

Promote the decentralization of core Effective Altruism organizations and projects into separate entities to distribute leadership, reduce single points of failure, and foster more focused responsibilities. This enhances resilience and clarity.

11. Plan a Structured AI Pause

Advocate for a defined “structured pause” in frontier AI development at the onset of an intelligence explosion (e.g., when AI significantly automates AI research). This pause would allow for critical deliberation and international cooperation on future governance.

12. Define Digital Being Rights Early

Proactively determine the moral considerations and rights (welfare, economic, political) that should be granted to digital beings before they are integrated into society. This is crucial because changing norms and legal rules will be very difficult once established.

13. Govern Post-AGI Resource Allocation

Establish governance mechanisms for newly valuable resources (e.g., energy, space) that will emerge after an intelligence explosion. This prevents a small number of actors from seizing control and potentially dominating Earth and the solar system indefinitely.

14. Prevent AI-Enabled Power Seizure

Develop political and governance strategies to prevent small numbers of actors, such as individuals or specific countries, from seizing extreme power through advanced AI technologies. This addresses the risk of destabilizing democracies and enabling dictatorships.

15. Safeguard Core EA Principles

Actively protect and promote the fundamental ideas of Effective Altruism, such as scope sensitivity, empathy for all beings, and an intense desire to use reason to help others. These principles are vital for the movement’s long-term impact.

16. Stand Firm on EA Principles

Encourage the Effective Altruism community to double down on its core principles and be willing to defend them, even in the face of significant attack, criticism, or unfair scrutiny. This fosters resilience and commitment to the movement’s mission.