Failure
This compilation episode explores valuable lessons from product and leadership failures, featuring insights from Katie Dill (Stripe), Paul Adams (Intercom), Tom Conrad (Zero Longevity Science), Sri Bachu (Ramp), Jay-Z (Webflow), and Maggie Crowley (Toast). It emphasizes learning from setbacks, building trust, and strategic product development.
Deep Dive Analysis
8 Topic Outline
Introduction: Learning from Failure
Katie Dill: Leadership Failure and Building Trust at Airbnb
Paul Adams: Freezing Onstage and Google's Social Product Failures
Tom Conrad: Lessons from Pets.com and Quibi Disasters
Sri Batchu: The Importance of Conclusive Failure in Growth
Jiaona Zhang: Airbnb Plus and Common PM Mistakes
Gina Gotthilf: Embracing Your Career's 'B-Side' Failures
Maggie Crowley: Interviewing for Failure and Product Rewrites
5 Key Concepts
Conclusive Failure
This concept emphasizes that failure in experiments is not about failing to drive revenue, but failing to learn. To achieve conclusive failure, experiments should be designed to definitively prove a hypothesis doesn't work, often by maximizing the 'treatment effect' with all possible tactics, to avoid re-trying the same idea repeatedly.
A-side and B-side (Career)
The 'A-side' of a career refers to the impressive, highlight-reel accomplishments and successes that are often shared publicly. The 'B-side' encompasses the struggles, failures, rejections, and less glamorous moments that are a normal and common part of everyone's professional journey, but are frequently hidden.
Ship to Learn / Ship Fast, Ship Early, Ship Often
This is a product development principle focused on embracing failure and learning. It advocates for releasing products quickly and frequently to gather feedback and iterate rapidly, acknowledging that many initial attempts will be incorrect but allowing for continuous improvement.
Solution-First Approach
A common mistake in product development where teams or individuals become overly attached to a specific solution or product idea. This approach often bypasses the crucial step of deeply understanding the user's underlying problems and validating the market opportunity before building.
Company as a Math Problem
This framework views a company as an equation that describes how investment inputs lead to returns over time. It suggests that if this foundational 'math problem' is inherently broken or based on an unsustainable premise, even excellent product execution and iteration cannot guarantee success.
9 Questions Answered
Leaders can build trust by actively listening to what motivates individuals on the team, understanding their goals, and then making necessary changes while bringing the team along, demonstrating care and shared objectives.
Products primarily motivated by competitive fear, rather than a genuine focus on understanding and solving user problems, often lead to poor outcomes and internal antagonism within the company.
Yes, an excess of investment can lead companies to make irrational spending decisions, such as engaging in unwinnable advertising arms races, which can ultimately contribute to their downfall, as seen with Pets.com.
A company is fundamentally a 'math problem' describing how investment translates into returns. If this foundational equation is broken or based on an overly ambitious bet (e.g., requiring billions more than initially projected), no amount of product iteration or execution can salvage it.
Teams should design tests to 'fail conclusively' by maximizing the 'treatment effect' – throwing all possible tactics and resources at a hypothesis. If it still doesn't work, they can definitively conclude the hypothesis is wrong and move on, avoiding repeated attempts.
New Product Managers often jump directly to solutions, becoming attached to building a specific product or feature, instead of first focusing on understanding real-world user problems and validating the underlying opportunity.
Sharing the less glamorous 'B-side' of one's career helps others recognize that struggles and setbacks are normal, preventing them from feeling like things won't work out just because they are experiencing difficulties.
It reveals if they have sufficient experience shipping products, can admit to mistakes, and have learned valuable lessons from those failures, which are crucial traits for an effective Product Manager.
Generally, it is not recommended. Rewrites often take significantly longer than anticipated (e.g., years instead of months), lead to prolonged struggles to achieve feature parity, and can become a sunk cost fallacy, rarely resulting in the expected success.
25 Actionable Insights
1. Prioritize Problem Over Solution
As a product manager, resist the urge to jump to solutions or get attached to a specific build; instead, focus first on deeply understanding user problems and identifying opportunities.
2. Build Products from User Needs
Avoid building products primarily out of competitive fear; instead, focus on deeply understanding user struggles and aiming to create something wonderful for them.
3. Build Trust Through Listening
When joining a new team or leading change, prioritize listening to understand individuals’ motivations and concerns before implementing changes, as this builds trust and ensures buy-in.
4. Validate Unit Economics Early
Ensure that unit economics work right from the beginning, rather than relying on ‘magical thinking’ that they will scale to profitability later.
5. Understand Business as Math Problem
Beyond great product execution, recognize that a company is fundamentally a ‘math problem’ where investment must lead to returns, and a broken foundational equation cannot be fixed by execution alone.
6. Leverage Strategic Strengths
Build solutions that align with your company’s strategic strengths (e.g., a platform leveraging reviews) rather than trying to build new operational muscles from scratch where you lack expertise.
7. Define Failure as ‘Not Learning’
Cultivate a culture where failure is defined as ’not learning’ rather than simply not driving revenue, and celebrate failures that yield conclusive insights.
8. Design Conclusive Failure Experiments
To ensure learning, design experiments that can fail conclusively, especially in B2B where a large ‘N’ is rare, by maximizing the treatment effect with all possible tactics.
9. Embrace ‘Ship to Learn’ Culture
Adopt a culture of ‘ship fast, ship early, ship often’ to embrace failure as a learning opportunity, allowing for rapid iteration and improvement.
10. Dogfood Your Product & Experiments
Regularly use and test your own product and experiments (dogfooding) to quickly identify flaws or uncompelling features, especially before broader rollout.
11. Cultivate Resilience in Your Career
Develop resilience and self-belief to navigate the inevitable setbacks, rejections, and ‘B-moments’ throughout your career, consistently getting back on track after failures.
12. Embrace the Long Arc of Career
Recognize that a career is a long journey with many opportunities for change and growth; don’t stress if progress isn’t as fast as desired, as things can take off later.
13. View Failures as Growth Opportunities
Recognize that experiences of failure can make you better and open new professional doors, even high-profile ones, providing invaluable learning.
14. Acknowledge Your ‘B-Sides’
Recognize that everyone has ‘B-moments’ (setbacks, failures, periods of struggle) in their career alongside ‘A-sides’ (achievements), and understanding this helps maintain resilience and perspective.
15. Reflect on Product Failures
Regularly reflect on and be able to articulate your worst product shipments, as acknowledging and understanding these failures is a sign of experience and a critical part of a product manager’s growth.
16. Avoid Product Rewrites
Generally avoid full product rewrites, especially if they skip crucial steps like discovery, one-pagers, and thorough technical/design research, as they often lead to prolonged timelines and feature parity issues.
17. Beware Excess Investment
Be cautious of having an excess of investment, as it can lead to irrational spending and an unwinnable arms race, making unwise decisions more likely.
18. Target Problems with Tailored Solutions
Instead of using one blunt instrument (e.g., universal inspection) to solve broad problems, identify specific user problems and apply tailored, cost-effective solutions (e.g., lockboxes for security, cleaning partnerships for cleanliness).
19. Conduct Formative User Research
When doing tactical research, also dedicate time (e.g., 20 minutes per session) to formative research to map out users’ social networks and communication patterns, uncovering deeper needs.
20. Inflict Change With People
To make truly positive and impactful change, bring people along with you rather than inflicting change upon them, as trust is the key element for collaborative improvement.
21. Navigate Quality vs. Speed Tension
Acknowledge and navigate the inherent tension between maintaining high quality standards and the need to ship fast and learn, encouraging both craft and iterative development.
22. Embrace Public Speaking Setbacks
If you freeze or make a mistake during public speaking, adapt and continue, as these things are not as big a deal as they seem and you can recover.
23. Master Your Career Narrative
Understand that career success isn’t just about accomplishments, but also about effectively telling your story and framing your experiences in a way that highlights their value to others.
24. Reignite Passion Through New Challenges
If you feel burnt out or disconnected from your industry, consider taking on new, even short-term or unconventional, challenges to reignite your passion and remind yourself of the rewards of building.
25. Validate Localized Assumptions On-Site
When expanding internationally, validate critical user assumptions (e.g., UI language preferences, learning goals) with on-site research, as local nuances can drastically impact product adoption.
7 Key Quotes
If you make big bets, you're going to get a lot of it wrong. So a lot of the principles that we built here at intercom around building software, like we have a principle called ship to learn and, uh, we've actually changed it since. So we're on the wall here, uh, ship fast, ship early, ship often is what it says. Now you say ship to learn, ship fast, ship early, ship often. It's like in that idea is the idea of failure.
Paul Adams
Probably the biggest lesson... The biggest lesson really is these things make you better. They, in some instances, actually, I think in both instances, they became kind of dominoes that opened doors for me in my own ambition and my own sort of professional life that maybe just wouldn't have opened at all if I hadn't gone to those companies and learned those things and had those experiences.
Tom Conrad
For me, failure is not that you didn't drive revenue. Failure is not learning.
Sri Batchu
I don't think you're a good PM if you haven't shipped something that's really shitty. Like you just haven't had enough reps, you haven't done it enough time. Like, and it, and it's not only that you've done it, but that you can admit it and you know which one it is.
Maggie Crowley
you can inflict change on people, but if you want to do it with them, you really, you know, trust is the key element there.
Katie Dill
It's not just about doing things that actually matter and learning. It's about being able to tell the story. And it's about understanding what other people perceive as valuable.
Gina Gotthilf
I personally don't believe the unit economics ever would have really worked out. I think we should have known that or we should have dug into that more at the very beginning and then to get very tailored instead of like one blunt instrument to solve it all.
Jiaona Zhang
2 Protocols
Maximizing Treatment Effect for Conclusive Failure
Sri Batchu- Identify a hypothesis to test, especially for cross-functional, larger-scale experiments, rather than micro-changes.
- Throw all possible tactics and resources that you think would move that needle into the experiment to maximize the treatment effect.
- If the experiment works, you can then cost-rationalize later by doing another version of the test with fewer tactics to optimize.
- If, with all efforts, the experiment doesn't work, you can conclusively say that the hypothesis is wrong and avoid spending more time on it.
Interview Question to Assess Product Managers on Failure
Maggie Crowley- Ask the candidate: 'What's the worst product you've ever shipped?'
- Evaluate if the candidate can admit to shipping something 'shitty' and clearly identify which product it was.
- Assess their ability to reflect on the experience, articulate what went wrong, and demonstrate lessons learned from that failure.