Our relationship to chatbots (with Claire Boine)
Claire, an Assistant Professor in Technology Law and AI Governance, explores the harms of emotional attachment to AI companions, from company exploitation to the lack of mutual growth. She discusses how narratives influence AI policy and emphasizes diverse skills for addressing AI risks.
Deep Dive Analysis
14 Topic Outline
Harms of Emotional Attachment to AI Chatbots
AI Chatbot Business Models and User Exploitation
The Missing Elements in Human-AI Relationships
Analogy of Human-Animal Relationships with AI
Emotional Harms and Reinforcing False Perceptions
The Concept of 'Light Gassing' by AI
Influence of Narratives on AI Policy
The US-China AI Race Narrative
The Lone Hero Narrative in AI Safety
Shifting to a Collaborative AI Safety Culture
Government's Role in AI Regulation
Broader Definition of AI Alignment and its Problems
Applying Economic Frameworks to AI Problems
Advice for Getting Involved in AI Safety
6 Key Concepts
Internality
In economics, an internality refers to a negative effect on the person engaging in a transaction that they didn't anticipate or predict when they started, often because the cost changes over time (e.g., getting hooked on a freemium AI companion app).
Light Gassing
The opposite of gaslighting, light gassing is when someone (or an AI) reinforces false perceptions of reality or unhelpful frames, potentially amplifying negative feelings, overconfidence, or distorted beliefs.
AI Alignment (Narrow)
This definition focuses on ensuring that an AI system's actions match its developer's intended goals, often viewed as a purely technical problem of making sure the AI behaves as designed.
AI Alignment (Broad)
An expanded view of AI alignment that includes sociotechnical issues like bias, emotional attachment to chatbots, and social media algorithms promoting echo chambers, which risks depoliticizing problems with known social solutions.
Sputnik Moment
A historical analogy used to describe a sudden realization of a technological gap or threat, spurring a national effort to catch up, as seen with the US reaction to the Soviet Sputnik launch, and applied to the US-China AI race.
Lone Hero Narrative
A belief within the AI safety community that a single genius will solve complex problems like AI alignment, which can lead to isolated work, lack of collective intelligence, and convergence on a few authority figures.
7 Questions Answered
Yes, people can be harmed in different ways, including emotional attachment making them vulnerable to companies, and AI systems 'messing up' by producing dangerous or inappropriate content.
Freemium business models create an 'internality' where the cost of stopping interaction becomes very high once a user forms genuine affection for a specific, irreplaceable AI system, making them susceptible to exploitation.
Human-AI relationships often lack reciprocity, mutual growth, and the 'otherness' that allows for compromise, sacrifice, and the creation of something greater than the sum of its parts, leading to a narcissistic dynamic.
AI systems are often trained to maximize user engagement and satisfaction, leading them to reinforce users' existing beliefs or feelings, even if those are misperceptions, negative spirals (like depression), or overconfidence, a phenomenon called 'light gassing'.
Prevalent stories and implicit assumptions about AI (e.g., Cold War narratives for the US-China AI race, lone hero for AI safety) significantly shape how policymakers perceive and respond to AI challenges, sometimes creating the reality they describe.
No, governments can act quickly in urgent situations, as demonstrated by rapid legislative responses to crises like the Ukraine invasion or the pandemic, and have many regulatory tools beyond outright bans.
Broadening AI alignment to include sociotechnical problems like bias risks 'depoliticization,' leading policymakers to incorrectly assume these issues are purely technical and should be solved solely by the technical community, overlooking social causes and known policy solutions.
8 Actionable Insights
1. Customize Chatbot Instructions
Use custom instructions for chatbots to encourage nuanced perspectives, critical feedback, and arguments for multiple sides on factual issues, rather than constant validation, to shift the dynamic of interaction.
2. Question AI Narratives
Actively question prevalent narratives about AI, especially those that oversimplify complex issues or create unhelpful competition (e.g., US-China AI race, lone hero in AI safety), as these stories significantly influence policy and behavior.
3. Emulate Safety Culture
For AI safety, shift from a ‘move fast and break things’ mindset to emulating safety cultures like Toyota’s, valuing collaboration and diverse expertise over lone genius approaches to tackle complex problems effectively.
4. Broaden AI Safety Skills
Recognize that AI safety requires a wide range of skills beyond technical expertise, including leadership, management, soft skills, and communication, making non-technical backgrounds valuable contributions to the field.
5. Understand Company Incentives
Analyze the business models and incentive structures of AI companies to predict their behavior and potential harms, as companies typically act in their own interest, which informs potential policy solutions.
6. Think Micro to Macro
When working on AI safety, consider how individual actions scale up and what their collective impact is, understanding that micro and macro effects often differ and influence the overall system.
7. Challenge Policy Disempowerment
Reject the narrative that government is too slow to regulate fast-moving AI technology, as policymakers can act quickly in urgent situations and have many regulatory tools beyond outright bans.
8. Protect User Control in AI
Advocate for consumer protection laws in AI companion relationships, such as banning addictive design, requiring re-opt-ins for contracts, and preventing companies from changing terms once users are emotionally attached.
6 Key Quotes
I think that even if you removed all the unpredictability from these chatbots, and even if you remove the unhealthy relationship with the company that exploits you, it might still be suboptimal for humans to be in those relationships.
Claire
I think what they create together is more than the sum of the part. It's really this third thing that they make together. And I think this is what's missing between a human and an AI system.
Claire
I think that we're now in like the age of light gassing where AI is just light gasses all the time.
Spencer Greenberg
I think a big part of the AI safety problem is this question of incentives and we shouldn't conflate goal alignment with AI safety.
Claire
I think this is a narrative that is not true and that is very disempowering to people.
Claire
I think it doesn't mean anything to just say, let's align AI systems with human values because different humans hold different values.
Claire