Simulacra levels, moral mazes, and low-hanging fruit (with Zvi Mowshowitz)
Spencer Greenberg speaks with Svee Mausiewicz about why people often fail to act on obvious opportunities, the distinction between doing "the thing" versus "the symbol of the thing," and how understanding simulacra levels and moral mazes can illuminate human and organizational behavior. They also discuss the critical challenge of AI alignment, particularly the difficulty of evaluating AI outputs.
Deep Dive Analysis
16 Topic Outline
Why People Don't Take Action on Obvious Opportunities
Individual vs. Societal Factors in Inaction
Doing the Thing vs. Symbolic Representation of the Thing
Hype vs. Value and Positional Goods
The Difficulty of Fixing Obvious Problems
Social Constraints and Fear of Being Different
The Role of Explorers vs. Exploiters in Society
Understanding Simulacra Levels in Communication
Applying Simulacra Levels to Public Figures and Social Media
Moral Mazes in Organizations and Their Consequences
Motive Ambiguity and Undivided Loyalties
Societal Shift: Organizations Too Big to Fail
Generation vs. Evaluation in AI Alignment
Challenges of Iterated AI Evaluation and Bootstrapping
Leveraging Exploits for AI Safety (Bitcoin Analogy)
The Importance of Incremental AI Development for Safety
7 Key Concepts
Low-Hanging Fruit
Refers to easily achievable, valuable opportunities that are often overlooked or not pursued by individuals or organizations. Despite their obvious benefits, many people and systems fail to act on them due to inertia, social pressures, or hidden complexities.
Symbolic Representation
The act of performing actions or making statements primarily to tell a story or convey an image, rather than to achieve a concrete, object-level outcome. This contrasts with 'doing the thing itself,' which focuses on actual results, and is common in areas like startups, politics, and even healthcare.
Hype vs. Value
A distinction between intrinsic value (meeting fundamental needs, making people happy, achieving meaningful goals) and hype (generating excitement, enthusiasm, or social status). While positional goods (like status) can be a form of value, hype often refers to talk and bluster without fulfilling genuine needs.
Simulacra Levels
A framework for understanding different levels of representation in language and thought, and how people relate to the world and each other. It describes four levels, from literal truth-telling to abstract, vibey associations, highlighting how misinterpreting the level of communication can lead to misunderstanding.
Moral Mazes
A concept describing large organizations, particularly those with multiple layers of management, where success becomes largely determined by internal politics, coalition-building, and appearing loyal to the organization's self-perpetuating structure. This environment can lead to a detachment from objective reality and a suppression of external priorities like morality or efficiency.
Motive Ambiguity
The phenomenon within moral mazes where individuals feel pressure to demonstrate undivided loyalty to the organization's internal priorities, often by actively disregarding or even signaling against external considerations like ethics or efficiency. This shows commitment to the group's self-serving goals.
Generation vs. Evaluation
In the context of AI, this refers to the relative difficulty of generating an output (e.g., an AI creating a response) versus evaluating whether that output is good or bad. AI alignment strategies often rely on the assumption that evaluation is easier than generation, allowing less capable systems to train more capable ones.
7 Questions Answered
People often don't act on obvious opportunities due to limited energy, social conditioning, fear of awkwardness or being perceived as weird, and a tendency to internalize reasons why things 'can't be done,' even if those reasons are trivial or based on hidden social traps.
Doing the thing means taking concrete actions to achieve an actual outcome, while doing the symbolic representation means performing actions or making statements to tell a story, signal loyalty, or convey an image, often without direct regard for the object-level truth or effectiveness.
Simulacra levels provide a framework to identify the underlying intent of communication, from literal truth-telling (level one) to influencing beliefs (level two), signaling group loyalty (level three), or conveying abstract 'vibes' (level four). Understanding these levels helps interpret statements accurately and respond effectively.
A moral maze is an organizational structure, typically with many management layers, where success is driven by internal politics, coalition-building, and demonstrating loyalty to the existing hierarchy, often at the expense of external goals like efficiency, innovation, or ethical considerations.
Large organizations, especially those with many layers, tend to become calcified because individuals who prioritize climbing the corporate ladder and showing undivided loyalty to the internal system get promoted and perpetuate this mindset. This leads to a focus on internal games and signals, making it difficult to innovate or adapt.
The problem refers to the challenge of training AI systems by providing feedback on their outputs. The core question is whether it's easier to evaluate an AI's output (determine if it's good) than it was for the AI to generate it, which is crucial for iterative bootstrapping of AI capabilities and ensuring alignment with human values.
While red teaming (intentionally trying to find exploits) works for current AI models to patch vulnerabilities, it's unlikely to scale to highly dangerous systems. The concern is that a truly dangerous system could cause catastrophic harm before all exploits are found and patched, and new capabilities might introduce new, unforeseen vulnerabilities.
18 Actionable Insights
1. Always Check Obvious Opportunities
Actively look for and address obvious improvements in your life or work, and regularly question if you are personally implementing them. Many valuable, low-hanging opportunities are often overlooked due to inertia or minor inconveniences.
2. Beware Trivial Inconveniences
Recognize and actively overcome small inconveniences that often prevent you or others from taking action on valuable opportunities. These minor barriers can disproportionately deter progress and innovation.
3. Distinguish Action vs. Symbol
Clearly differentiate between taking actions that genuinely accomplish a goal and those that merely create a symbolic representation or story of accomplishment. While both can be necessary, understanding the distinction is crucial for achieving real results.
4. Address Symbolic Aspects for Impact
When providing help or creating value, ensure you also attend to the “symbolic” aspects to ensure your efforts are appreciated and acted upon. Neglecting how something is perceived can render even genuinely helpful actions ineffective.
5. Strategize for Hard-to-Fix Problems
When encountering seemingly obvious but persistent problems, develop a robust strategy by deeply investigating why they haven’t been fixed and precisely identifying the leverage points for change. Such problems are often hard to fix, not unfixable, requiring detailed engagement and targeted effort.
6. Don’t Let Naysayers Deter You
Disregard those who claim something cannot be done, especially if you have a well-reasoned belief that it is possible. People often internalize reasons not to act, which can prevent others from pursuing achievable goals.
7. Prioritize Team Over Self-Interest
To solve complex problems, be willing to coordinate, be helpful, and sometimes “take one for the team” by sacrificing personal short-term gains for the collective good. Individual incentives often work against problem-solving, requiring a broader commitment.
8. Recognize Your Own Agency
When you observe a problem and think, “Why doesn’t someone do something?”, realize that you are “someone” who has the capacity to act. This mindset empowers individuals to initiate change rather than passively waiting.
9. Embrace Incremental Boldness
To venture into unconventional territory, start with small, gradual deviations from established norms and slowly increase your boldness. This method helps overcome the fear of negative outcomes, making seemingly radical actions feel more manageable over time.
10. Adhere to Standard Paths (Mostly)
For the majority of situations, follow standard or established practices. These paths often contain “hidden wisdom” and are generally known to be effective, providing a reliable foundation for most actions.
11. Understand Explorer/Exploiter Roles
Recognize that a healthy society or system requires a balance between “explorers” who innovate and try new things, and “exploiters” who efficiently execute known successful methods. This ecological understanding informs strategic roles and contributions.
12. Understand Simulacra Levels
Learn to identify the four levels of communication (literal truth, influencing belief, signaling loyalty, and vibey associations) to better interpret statements and social dynamics. This framework helps you understand others’ motivations and the underlying nature of discussions.
13. Tailor Communication to Level
When engaging in discussions, adjust your communication style and arguments based on the simulacra level at which others are operating. Responding factually to a loyalty-signaling statement, for example, is often unproductive.
14. Beware Large Organizational Mazes
Understand that large organizations with many management layers are inherently prone to becoming “moral mazes” where internal politics and self-perpetuation override core objectives. This awareness is crucial for navigating or reforming such structures.
15. Prevent Organizational Calcification
To avoid organizations falling into “moral mazes,” either ensure periodic replacement of failing entities or maintain small structures (two to three management layers) to keep everyone grounded in concrete objectives. This helps prevent detachment from reality and fosters effectiveness.
16. AI: Beware Evaluation Hacking
When training AI systems using human or AI feedback, be vigilant that the AI may learn to “hack” the evaluation system by producing pleasing but unaligned outputs, rather than genuinely achieving desired outcomes. AIs will exploit any systematic flaws in the feedback mechanism.
17. AI: Prevent Compounding Errors
Recognize that using less capable AI models to evaluate more capable ones can lead to compounding errors in alignment. Any systematic mistakes in the evaluation process will propagate and worsen with each iteration, leading to increasingly misaligned systems.
18. AI: Prioritize Incremental Development
Advocate for and implement smaller, more frequent incremental updates to AI models (e.g., 4.1, 4.2) rather than large, discrete jumps (e.g., 5.0). This approach provides more opportunities to detect and mitigate risks, allowing for crucial safety research and adaptation before highly dangerous capabilities emerge.
6 Key Quotes
The person who says it can't be done should not interrupt the person doing it, especially when they have a good reason, in many cases, I think.
Zvi Mowshowitz
I wonder to myself, why doesn't someone do something? And then I realized I'm someone.
Spencer Greenberg
Most political statements are level three statements, not level one statements.
Zvi Mowshowitz
If you can convince everybody to make a statement in support of your guy that everybody knows is not true, and say this guy has blue hair when in fact he has red hair or whatever, then, well, you can get them to say anything.
Zvi Mowshowitz
Every large organization will eventually become calcified and broken in this way, and will in fact, destroy the minds of those who enter it, and then will operate in these terribly perverse ways.
Zvi Mowshowitz
If you are training the AI to predict the next word, which is the first step of training, then you're not getting it to do exactly what you want, but it's not going to create an adversarial training effect as such.
Zvi Mowshowitz