Why great AI products are all about the data | Shaun Clowes (CPO Confluent, ex-Salesforce, Atlassian)

Dec 29, 2024 1h 21m 13 insights Episode Page ↗
Sean Klaus, CPO at Confluent, discusses why most PMs aren't great and how to become a 10x PM by focusing externally and being data-informed. He shares insights on AI's impact on product management, emphasizing data management, building effective B2B growth teams, and a "bingo card" approach to career development.
Actionable Insights

1. Prioritize External Focus for PMs

Always initiate discussions and documents from the customer, market, and competitor perspectives. This approach is crucial for finding reliable, differentiated value and gaining internal support for your product decisions.

2. Be Data-Informed, Not Driven

Use data as a compass to disprove assumptions and identify opportunities, rather than expecting it to provide definitive answers. This means supporting your statements with data, but also trusting your intuition when data seems counter-intuitive.

3. Rigorous Qualitative Research with LLMs

Right-size your customer interview efforts (interviewing 7-14 people to learn new things), avoid asking leading questions, and leverage LLMs to find where your strategy doesn’t align with customer feedback or where competitors’ positioning might fit better.

4. Leverage LLMs for Feedback Synthesis

Utilize AI tools to process and summarize vast quantities of inbound customer requests, identifying common themes, popular ideas, and trends in demand across hundreds or thousands of pieces of feedback for deeper insights.

5. Prioritize Data Management for AI

When building AI-powered products or integrating AI into workflows, focus 90% of your effort on acquiring, structuring, and delivering timely, high-quality, and relevant data to the LLM. The model’s effectiveness is directly proportional to the quality and recency of the data it receives.

6. Verify Counter-Intuitive Data

If data results contradict your intuition, trust your intuition first, then rigorously investigate the data’s representativeness, selection bias, upstream/downstream context, and broader business impact (e.g., average sale price) before accepting the findings.

7. Manage Calendar for External Focus

Actively manage your calendar to allocate sufficient time (e.g., 80%) for external thinking and customer/market analysis. This prevents internal politics or delivery tasks from consuming all your time, which is essential for successful product management.

8. Optimize Decision-Making with Data

Aim to make decisions with between 30% and 77% of available data. Less than 30% is too risky, and waiting for more than 77% means you’ve waited too long, leading to missed opportunities.

9. Build a ‘Bingo Card’ Career

Intentionally choose diverse roles and experiences that fill gaps in your skill set (e.g., different sales models, product types, company sizes). This approach makes you a more versatile professional, equipping you with ‘superpowers’ to tackle varied problems.

10. Understand the Whole Business

Cultivate a broad understanding of various company functions (e.g., finance, legal, sales) beyond your direct responsibilities. This enables you to contribute strategically and ‘be dangerous’ (in a complimentary way) in diverse business situations.

11. Balance PLG with Sales Motion

For B2B companies, strive to make both Product-Led Growth (PLG) and traditional sales motions work together. This creates a resilient business model by leveraging PLG to feed sales and sales to inform PLG, resulting in a large customer base and high revenue.

12. Prioritize Trust and Relationships

Recognize that people are more receptive to your ideas and advice when they feel you genuinely care about them and their outcomes. Building trust and strong relationships forms the foundation for effective influence and collaboration.

13. Be Realistic About Product Failure

If a product is clearly failing (e.g., after six months), be honest with yourself and stakeholders, and advocate for killing it early. This prevents it from becoming an accidental, long-term drain on resources and allows for reallocation to more promising ventures.