Inside Google's AI turnaround: The rise of AI Mode, strategy behind AI Overviews, and their vision for AI-powered search | Robby Stein (VP of Product, Google Search)
Robbie Stein, VP of Product for Google Search, discusses Google's AI transformation, including Gemini's success and AI mode. He shares lessons on product building, emphasizing relentless improvement, deep user understanding, data-driven decisions, and the importance of humility and curiosity.
Deep Dive Analysis
18 Topic Outline
Google's AI Momentum and Gemini's Success
Google Search's Resilience and AI's Expansionary Role
Deep Dive into Google's AI Mode and its Capabilities
The Shift to Natural Language and AI's Impact on Content
Google AI's Unique Information Retrieval Approach
Lessons from Building Successful AI Products
Robby Stein's 'Relentless Improvement' Philosophy
Strategies for Driving Growth in Existing Products
Instagram Stories: Learning from Competitors
The Rapid Development and Launch of Google AI Mode
Google's Urgency in AI Product Delivery
Positioning Google AI Mode Against Other Chatbots
Three Core Product Principles for Success
Case Study: Instagram Close Friends' Long Road
The Importance of Sufficient Resources for Breakthroughs
AI Corner: Multimodal AI for Visual Inspiration
Parenting in the AI Era: Kids Using Conversational AI
Startup Story: Getting Justin Bieber on the Stamped App
6 Key Concepts
Relentless Improvement
This philosophy involves being the physical manifestation of two things: complete, positive effort and a constant drive to make things better. It means never being content and always striving to improve, acting as the harshest critic of one's own work.
AI as Expansionary
AI is not replacing core search functions but rather expanding the scope of what users can ask and discover. It fulfills more questions and curiosities, leading to new use cases like visual search, thereby growing the overall interaction with search products.
Query Fan Out
When Google's AI generates a response, it acts as a tool to perform numerous background searches (dozens of queries) using Google Search. This process allows the AI to access real-time information and make requests to various data backends to construct a comprehensive answer.
Jobs to Be Done Framework
A mental model that encourages understanding why a user 'hires' a product to accomplish a specific task or fulfill a need, rather than just observing product usage. It focuses on the causation behind product adoption, including both functional utility and emotional drivers.
Clarity Over Cleverness (Design Principle)
This principle advocates for designing products with clear, intuitive interfaces that leverage existing user mental models and standards, rather than striving for unique or overly clever designs. The goal is to avoid user confusion and ensure easy adoption by making functions immediately understandable.
J-Curve (Product Retention)
A metric used to track the percentage of users who continue to use a product over specific time intervals (e.g., day 7, day 30, day 90). It helps determine if a product has sustained user engagement (retention flattens out) or if users are consistently dropping off over time.
10 Questions Answered
There's an incredible sense of focus and urgency to deliver great products quickly, with product teams and research groups working very closely together, leading to a compounding effect of ruthless improvement that has hit a tipping point.
No, the core Google Search isn't changing; people still come for a vast array of specific informational needs. AI is expansionary, leading to more questions and curiosity being fulfilled, rather than replacing traditional search.
AI mode is specifically designed for informational tasks and search, leveraging Google's vast knowledge graph (products, places, web context) and real-time data, and is less focused on creativity or productivity tasks like spreadsheet analysis.
AI models perform background searches and evaluate content based on helpfulness, sources, originality, and user intent, similar to traditional SEO. Creators should consider what kind of content people use AI for (e.g., advice, how-to, complex needs) and make their content best for those needs.
Teams often give up too early or under-invest in the product, especially for hard technical problems, because they adhere too strictly to a 'scrappy, lean' mentality, preventing the product from ever becoming good enough to gain momentum.
Instagram recognized that the ephemeral story format was critical for users to share their lives and connect with less pressure, and saw it as a format to make their own, rather than inventing something entirely new, ultimately providing a better product experience for their users.
Deeply understand why people use the product (its true essence or 'job to be done'), identify new growth drivers or formats that are complementary rather than replacement, and carefully integrate new features to feel coherent yet distinctive for your specific user base.
Prioritize clarity over cleverness, as users expect certain standards and mental models (e.g., a camera icon should look like a camera). Reinventing familiar interfaces can confuse users and work against the product's adoption.
The initial version failed due to user confusion (e.g., mistranslated as 'best friend' leading to small lists, unclear design). Success came from deeply understanding the emotional job (connecting with friends), simplifying the product to only stories, and making the design (green ring) clearly communicate its private nature.
Encourage kids to use live, conversational AI tools (like Google's voice-activated search) to ask questions and learn about anything they're curious about, as this natural interaction helps them adapt to AI as a learning tool.
20 Actionable Insights
1. Embody Relentless Improvement
Exert complete, positive, and productive effort, always striving to make things better and never being content with the status quo. This mindset drives continuous improvement and eventually leads to a tipping point of usefulness.
2. Cultivate Humility
Maintain a constant state of humility, questioning your own assumptions, actively listening to users and others, and being open to admitting when you are wrong. This mindset fosters continuous learning and better product decisions.
3. Study User Causation
Focus on understanding the “job to be done” – the underlying problem or desire that causes a user to “hire” your product, rather than just what they are “using.” This deep understanding of causation helps build products that truly meet user needs.
4. Leverage Metrics for Root Cause
Use analytical tools like retention J-curves and core metrics as a guide to understand product performance. If metrics drop, conduct root cause analysis to pinpoint specific issues (e.g., region, device, demographic, use case) to make data-driven decisions for improvement.
5. Prioritize Clarity in Design
Opt for clear, intuitive design that leverages existing user understanding and standards, rather than striving for clever or overly differentiated aesthetics. Clarity in design minimizes user confusion and maximizes product usability.
6. Challenge Daily Tolerances
Actively question and challenge everyday inconveniences or “sucky” experiences rather than tolerating them, asking “why” and “how to make it better” to drive innovation. This mindset helps identify opportunities for improvement in products and life.
7. Grasp Product’s Core Purpose
Deeply understand the fundamental reason users engage with your product and its true essence (e.g., sharing life, connecting with people) to identify existential threats or opportunities for new formats. This understanding guides strategic product evolution.
8. Adopt External Innovations
Recognize that not all great ideas will originate internally and be willing to adopt successful formats or features from competitors to provide the best possible product experience for your users. Failing to do so denies users a better product.
9. Integrate New Features Thoughtfully
When adopting new features or formats, ensure they are adapted to your product’s unique identity, user expectations, and use cases, rather than simply copying them. This involves making careful design decisions to ensure coherence while maintaining distinctiveness.
10. Shift Resources from Diminishing Returns
Continuously evaluate projects for diminishing marginal returns; when further investment yields little impact, reallocate resources to new growth drivers or initiatives that address fundamental market shifts. This ensures efficient resource allocation and sustained growth.
11. Scale Teams on Product Conviction
Begin with a small, scrappy team to prove out a concept and build internal conviction, supported by early external validation from trusted testers. Once conviction is strong, invest sufficiently in resources to build the best possible version for launch, rather than keeping the team too small for too long.
12. Utilize Trusted Tester Groups
Before a wider launch, engage a trusted tester group (e.g., 500 external users, friends, and family) to gather honest, critical feedback on product functionality and user experience. This approach helps identify flaws early and iterate quickly, similar to a startup.
13. Observe User Search Behavior
Scrutinize user query data for patterns, like users appending “AI” to their searches, to identify unmet needs and build new products that directly address those explicit user problems. This helps in understanding what users are trying to achieve.
14. Address Emotional User Needs
When identifying user “jobs to be done,” recognize and address both utility-based and emotional needs, as emotional drivers are often overlooked but critical for product success. Understanding these deeper motivations can lead to more impactful solutions.
15. Optimize for User Connection
Design features to facilitate genuine connection and interaction among users, especially in social products, ensuring that the core “job” of connecting with friends is successfully completed (e.g., by ensuring replies or engagement). If this loop is broken, the product fails.
16. Foster Intense Curiosity
Cultivate an intense curiosity to understand the “why” behind everything – from user behavior to differing opinions and product failures – and relentlessly pursue knowledge until you fully comprehend. This innate drive is crucial for effective problem-solving and innovation.
17. AI-Assisted Source Discovery
Leverage AI as a “curiosity engine” to discover information and “cool links” to original sources, such as old papers and PDFs, rather than solely relying on AI-generated summaries. This hybrid approach combines AI’s discovery power with deep, foundational learning.
18. Leverage AI for Visual Inspiration
Utilize AI for multimodal visual and inspirational tasks, such as generating image boards for design ideas (e.g., office decor) and engaging in multi-turn conversations to refine visual preferences. This expands AI’s utility beyond text-based tasks.
19. Promote AI-Native Learning
Encourage children to use live, conversational AI tools (like Google Live) for natural language learning, allowing them to directly ask questions about various topics. This helps them become “AI native” and leverages an accessible learning modality.
20. Prioritize Intense Urgency
When pursuing opportunities, act with intense urgency and immediacy, being scrappy rather than overthinking or delaying. This approach often leads to breakthroughs and successful outcomes.
8 Key Quotes
AI is expansionary. There's actually just more and more questions being asked and curiosity that can be fulfilled now with AI.
Robby Stein
You need to be the physical manifestation of two pieces of things. One is just relentlessness, like just complete effort that is always exerted in a direction of positive productivity. And the second is make things better. You have to always make things better. You're never content.
Robby Stein
Not every great thing is going to be invented by you.
Robby Stein
At the end of the day, you're kind of just robbing your user base of an opportunity to have a better product.
Robby Stein
I view product like golf, like you're always one stroke away from shanking. And like, as soon as you think you're good, you're not like, you don't know anything.
Robby Stein
People don't want a quarter inch drill. They want a quarter inch hole.
Robby Stein
If you try to build the most beautiful, symmetric two handles on each side on a glass door, it like doesn't communicate in for any information to you.
Robby Stein
Intense urgency usually wins over thinking about it for a long time.
Robby Stein
1 Protocols
Protocol for Building Successful Products
Robby Stein- Deeply understand people: Be a student of causation, understanding why someone 'hires' your product (utility and emotional jobs to be done). Use techniques like 'interrogation' interviews to find the 'big hire' moment.
- Analytical rigor and understanding your problems: Use instrumentation (e.g., J-curve retention) to know if you're on the right track. Conduct root cause analysis when metrics drop to understand 'why' and identify the specific problem (e.g., region, device, demographic, use case).
- Design for clarity instead of cleverness: Prioritize clear communication and lean into established standards and mental models that users already understand. Avoid reinventing interfaces unless absolutely necessary, and ensure designs convey information effectively.
- Be humble: Constantly question yourself, listen to others and users, and be open to being wrong.