Why great AI products are all about the data | Shaun Clowes (CPO Confluent, ex-Salesforce, Atlassian)
Sean Klaus, CPO at Confluent, discusses why most PMs aren't great and how to become a 10x PM by focusing externally and being data-informed. He shares insights on AI's impact on product management, emphasizing data management, building effective B2B growth teams, and a "bingo card" approach to career development.
Deep Dive Analysis
11 Topic Outline
The Undeveloped State of Product Management
Becoming a 10x Product Manager
Leveraging AI for Product Management Insights
AI's Impact on Data Management for Product Building
AI's Disruption of Enterprise SaaS Businesses
Reformed Perspective on Data-Driven Decision-Making
Building and Scaling B2B Growth Teams
The Evolution and Importance of Product-Led Growth
Shaun Clowes' Career Decision-Making Philosophy
Lessons from a Product Failure
Final Thoughts and Career Advice
6 Key Concepts
10x Product Manager
A product manager who enables others to have dramatically more impact, leading to a 100x or greater return on resources. This role focuses on finding reliable, differentiated value rather than internal politics or delivery.
Feedback River
A concept where smart product managers continuously immerse themselves in various forms of feedback, including user interviews, direct customer feedback, NPS data, and competitor information, to gather insights.
Information Decay Rate
The rate at which the value of new data, such as customer feedback or competitor actions, diminishes over time for decision-making. LLMs require timely data due to this rapid decay.
Business Rules (SaaS Value)
The core value and lock-in of enterprise SaaS applications like Salesforce or Workday come not from their UI or data models, but from the years of configured underlying workflows and processes that make the application specifically tailored to a company's operations.
Data as a Compass
A mental model suggesting that data should serve as a guide to disprove assumptions or indicate potential opportunities, rather than providing definitive answers. Treating data as a GPS (giving exact answers) can lead to being slow or wrong.
Bingo Card Career Philosophy
Shaun's approach to career development, where he intentionally chooses diverse roles and experiences to "fill in boxes" of new knowledge and skills. This strategy aims to increase versatility and provide unique perspectives for future challenges.
10 Questions Answered
Product managers frequently get sidetracked by internal politics, scrum management, or delivery tasks, diverting their focus from the crucial external perspectives of customers, markets, and competitors, which are essential for identifying differentiated value.
To level up, PMs should dedicate about 80% of their time to thinking from outside the building (customer, market, competitor viewpoints) and be data-informed, using data to substantiate their claims and build consensus.
PMs can use LLMs to identify discrepancies between their strategy and customer feedback, determine if customer statements align better with competitors' offerings, and semantically group thousands of inbound requests to pinpoint trending demands.
The most significant impact of AI will be on data management. LLMs are only as effective as the timely, high-quality data they receive, meaning product managers must prioritize gathering and structuring comprehensive data to feed these models.
No, the core value and "lock-in" of these enterprise SaaS applications come from their complex, configured business rules and workflows that evolve over years to fit a specific company's processes, rather than just their user interface or data model.
Data should function as a compass, guiding intuition and helping to disprove assumptions, rather than a GPS that provides exact answers. Over-relying on data for every decision can lead to being slow or making incorrect choices, as intuition often synthesizes vast amounts of past data.
Growth teams typically progress through phases of proving initial value (the "gold rush"), scaling repeatable processes, and integrating with other organizational functions. Challenges include maintaining momentum after initial wins, systematizing growth, and managing complex relationships across departments.
PLG is crucial because it creates a natural force for companies to prioritize end-user enjoyment and success, which might otherwise be overlooked due to focus on economic buyers. When combined with sales, PLG can build a resilient business with both a large customer base and substantial revenue.
Adopt a "bingo card" approach, deliberately selecting roles that offer new, diverse experiences and challenge existing skill sets. This strategy fosters versatility, provides unique perspectives, and equips one with "superpowers" to tackle future problems.
Always question the data's representativeness, sample quality, and potential biases. Before acting, examine what happened upstream and downstream, and consider the broader context (a "click up") to ensure the insights are truly meaningful and not just random aberrations or temporary benefits.
13 Actionable Insights
1. Prioritize External Focus for PMs
Always initiate discussions and documents from the customer, market, and competitor perspectives. This approach is crucial for finding reliable, differentiated value and gaining internal support for your product decisions.
2. Be Data-Informed, Not Driven
Use data as a compass to disprove assumptions and identify opportunities, rather than expecting it to provide definitive answers. This means supporting your statements with data, but also trusting your intuition when data seems counter-intuitive.
3. Rigorous Qualitative Research with LLMs
Right-size your customer interview efforts (interviewing 7-14 people to learn new things), avoid asking leading questions, and leverage LLMs to find where your strategy doesn’t align with customer feedback or where competitors’ positioning might fit better.
4. Leverage LLMs for Feedback Synthesis
Utilize AI tools to process and summarize vast quantities of inbound customer requests, identifying common themes, popular ideas, and trends in demand across hundreds or thousands of pieces of feedback for deeper insights.
5. Prioritize Data Management for AI
When building AI-powered products or integrating AI into workflows, focus 90% of your effort on acquiring, structuring, and delivering timely, high-quality, and relevant data to the LLM. The model’s effectiveness is directly proportional to the quality and recency of the data it receives.
6. Verify Counter-Intuitive Data
If data results contradict your intuition, trust your intuition first, then rigorously investigate the data’s representativeness, selection bias, upstream/downstream context, and broader business impact (e.g., average sale price) before accepting the findings.
7. Manage Calendar for External Focus
Actively manage your calendar to allocate sufficient time (e.g., 80%) for external thinking and customer/market analysis. This prevents internal politics or delivery tasks from consuming all your time, which is essential for successful product management.
8. Optimize Decision-Making with Data
Aim to make decisions with between 30% and 77% of available data. Less than 30% is too risky, and waiting for more than 77% means you’ve waited too long, leading to missed opportunities.
9. Build a ‘Bingo Card’ Career
Intentionally choose diverse roles and experiences that fill gaps in your skill set (e.g., different sales models, product types, company sizes). This approach makes you a more versatile professional, equipping you with ‘superpowers’ to tackle varied problems.
10. Understand the Whole Business
Cultivate a broad understanding of various company functions (e.g., finance, legal, sales) beyond your direct responsibilities. This enables you to contribute strategically and ‘be dangerous’ (in a complimentary way) in diverse business situations.
11. Balance PLG with Sales Motion
For B2B companies, strive to make both Product-Led Growth (PLG) and traditional sales motions work together. This creates a resilient business model by leveraging PLG to feed sales and sales to inform PLG, resulting in a large customer base and high revenue.
12. Prioritize Trust and Relationships
Recognize that people are more receptive to your ideas and advice when they feel you genuinely care about them and their outcomes. Building trust and strong relationships forms the foundation for effective influence and collaboration.
13. Be Realistic About Product Failure
If a product is clearly failing (e.g., after six months), be honest with yourself and stakeholders, and advocate for killing it early. This prevents it from becoming an accidental, long-term drain on resources and allows for reallocation to more promising ventures.
6 Key Quotes
People don't care what you know until they know that you care.
Shaun Clowes
LLMs can only be as good as the data they are given and how recent that data is.
Shaun Clowes
The job is not about execution or anything. It's about finding reliable, differentiated value, right? That you, that you can uniquely deliver into the market.
Shaun Clowes
People really underestimate where the value is created in these applications and they just kind of get it completely wrong.
Shaun Clowes
Your job is to say no to 90% of things that get brought your way.
Shaun Clowes
Show me the incentive and I'll show you the outcome.
Charlie Munger (quoted by Lenny)