The path to utopia (with Nick Bostrom)

1. Improve Data & Cultural Backups

Actively work to improve the backup and preservation of data and cultural information on both individual and civilizational scales. This is crucial for safeguarding knowledge against loss, ensuring continuity, and providing valuable resources for future generations.

2. Align Systems with Correct Goals

When designing or managing complex adaptive systems (like AI, companies, or societies), ensure their objectives are correctly set. Misaligned goals can lead to powerful optimization abilities being directed towards dangerous or undesirable outcomes.

3. Develop Civilizational Coordination Structures

To enable humanity or civilization to act virtuously, focus on creating effective coordination structures. Virtue requires understanding a situation and deliberately choosing good actions, which for a collective entity, necessitates robust coordination.

4. Cultivate Group & Civilizational Virtues

Recognize that virtues can be ascribed to groups and entire civilizations, not just individuals (e.g., intellectual honesty in a team, environmental care in society). Actively work towards cultivating these emergent virtues for collective benefit.

5. Carefully Consider Spreading Life

Before actively spreading life across the universe (e.g., via panspermia), carefully consider the ethical implications, particularly the potential for generating vast amounts of suffering. It’s advisable to “think this through rather carefully” due to the extremely high stakes.

6. Use Limits for Future Planning

When thinking about the long-term future, use fundamental limits (e.g., laws of physics) as guideposts to establish rigorous boundaries for what’s possible. This approach helps ground speculation and make predictions more robust and decision-relevant.

7. Assess Civilizational Energy Budget

To understand the potential activities and limits of a civilization, assess its total mass-energy budget and the amount of useful energy that can be extracted. This provides a fundamental bound on what can be achieved.

8. Consider Black Holes for Energy

For long-term energy planning, consider black holes as an ultimate energy source, capable of extracting a vast amount of mass-energy from old stars even after they’ve died. This perspective opens up possibilities for civilization far beyond stellar lifetimes.

9. Improve Services & Computing Efficiency

Focus on improving the efficiency of services and computing, as current systems are far from thermodynamic limits and could operate with significantly less energy and better organization. This can lead to decoupling economic growth from high energy use.

10. Reduce Avoidance via Well-being

Cultivate a “fundamental sense of well-being” or “deep okay-ness” to potentially reduce avoidance behaviors driven by fear or anxiety. This mindset could empower individuals to pursue valuable opportunities and actions that might otherwise be intimidating.

11. Strive for Clarity in Arguments

When engaging in discussions, especially on sensitive topics like ethics, recognize that arguments are often biased by cultural views or “yuck reactions.” The best approach is to be as clear as possible about your own assumptions and reasoning, even if true unbiasedness is unattainable.

12. Explicitly State Intuition Basis

In philosophical or critical thinking, explicitly acknowledge when arguments are based on intuition rather than empirical evidence or formal logic. This transparency helps clarify the strength and nature of the argument.

13. Test Analogies for Misleadingness

When using analogies to explain complex concepts, rigorously test them on diverse audiences to identify where they might mislead or generate incorrect conclusions. Be prepared to discard or refine analogies that cause significant confusion.

14. Correct Misconceptions Gently

When encountering misconceptions, especially those stemming from oversimplified explanations, strive to correct them gently and clearly. Frame the accurate explanation as something “even more awesome” to encourage understanding rather than defensiveness.

15. Take Counter-Intuitive Math Seriously

Be prepared to take mathematical predictions seriously, even when they suggest counter-intuitive or “weird” phenomena, as history shows that such predictions (e.g., antiparticles) can turn out to be true.

16. Distinguish Model Errors from Reality

Cultivate the ability to discern whether a model is simply broken and producing nonsense, or if it’s accurately reflecting a genuinely weird aspect of reality. This critical distinction is a hallmark of quality thinking.

17. Foster Interdisciplinary Alignment Collaboration

To address the alignment problem in complex systems (like AI or societal structures), foster interdisciplinary collaboration between fields such as AI safety, economics, and political science. This can lead to innovative solutions for incentive design and governance mechanisms.

18. Seek Robust, Noise-Free Explanations

Strive for explanations that are robust and hold true regardless of personal beliefs, avoiding those that merely justify pre-existing preferences. Use analogies carefully to avoid introducing “noise” or misleading interpretations.

19. Beware Jevon’s Paradox Effects

Be aware that making resources cheaper (e.g., energy) often leads to increased consumption rather than savings, a phenomenon known as Jevon’s paradox. This understanding can help anticipate and mitigate unintended consequences of efficiency improvements.

20. Balance Green Initiatives, Speed

When pursuing “green” initiatives, be mindful of potential trade-offs, particularly between energy efficiency (e.g., lower temperature processes) and operational speed. Sometimes, achieving high efficiency might require sacrificing quick results.

21. Recognize Biological Trade-offs

Understand that biological systems, like wound healing or cell division, involve trade-offs between speed and safety (e.g., rapid division risking cancer). This highlights a general principle of balancing immediate benefits with long-term risks.

22. Prioritize System Resource Allocation

Acknowledge that within any complex system (like the body or an organization), different components will compete for resources. Effective management involves setting priorities and understanding these internal “negotiations” to optimize overall system function.

23. Manage Stress for Immune Health

Be aware that long-term stress can reduce the immune system’s share of resources, potentially weakening it, while short-term stress may increase it. Managing stress levels is crucial for maintaining a healthy immune response.

24. Engage with Doubts to Learn

Even when confident in a theory or limit, engaging with those who doubt it can be highly beneficial. This discourse can lead to important new learnings and a deeper understanding of the underlying principles.

25. Anticipate Surprises in Limits

While relying on known limits for future planning, remain open to the possibility that current understandings of natural laws and limits may be proven wrong. Expect “nasty surprises” and adapt, as our knowledge is always evolving.

26. Optimize Computing Speed vs. Reversibility

When designing computing systems, recognize the trade-off between energy efficiency (achieved by reversible operations) and speed. Practical applications often necessitate using some irreversible operations to achieve results quickly, even if it incurs an energy cost.

27. Account for Error Correction Costs

When considering energy budgets for advanced computing, remember that error correction is an inherently irreversible and energy-intensive operation. This cost will likely be a significant factor even in otherwise perfectly reversible systems.