Career frameworks, A/B testing mistakes, counterintuitive onboarding tips, selling to developers | Laura Schaffer (VP of Growth at Amplitude)
Laura Schaefer, Head of Growth at Amplitude, shares her career growth framework, emphasizing proactive customer insight gathering. She also discusses surprising experimentation insights, the importance of user psychology in growth, and strategies for Product-Led Growth and selling to developers.
Deep Dive Analysis
15 Topic Outline
Laura Schaffer's Career Growth Framework
The Value of Customer Insights and Voice of the Customer Reports
Proactive Career Building and Leveraging Strengths
Unexpected Experiment Results: Good Friction Improves Conversion
Understanding the User's Psyche in Product Onboarding
Iterative Experimentation vs. Full Redesigns
Strategies for Faster Experiment Validation
Rethinking Confidence Intervals in Experimentation
The Importance of Long-Term Growth Team Expectations
Twilio's 'Quick Deploy' for Non-Developers
Integrating Product-Led Growth (PLG) with Sales-Led Growth (SLG)
Unique Characteristics of Selling to Developers
Recommended Books for Happiness and Habits
Favorite Interview Question and Product Development Tip
Leveraging Amplitude with Other Tools
8 Key Concepts
Proactive Career Growth
Instead of waiting for managers to define a career path, individuals should proactively seek opportunities by identifying customer insights and problems, then proposing solutions to leadership. This builds a personal brand and creates opportunities beyond a predefined role.
Voice of the Customer Report
A digest of customer insights and pain points, compiled and shared regularly within a company. This helps senior leaders stay connected to customer needs and can establish the compiler as a subject matter expert, leading to new initiatives.
Good Friction vs. Bad Friction
Not all friction in a user flow is detrimental; some can be beneficial. Good friction addresses user anxieties or provides reassurance, making the user feel more confident and understood, which can surprisingly improve conversion rates.
User Psyche in Onboarding
Understanding the mental state of users, especially when encountering a product for the first time, is crucial. Users often anticipate difficulty or fear being out of their depth, and addressing these psychological barriers can be more impactful than simply removing steps.
Iterative Experimentation
A strategy of making small, incremental changes and testing them frequently, rather than launching large, complete redesigns. This approach acknowledges that most hypotheses fail, allowing teams to learn quickly and use failures as a 'compass' to guide future development.
80% Experiment Failure Rate
A statistical observation that roughly 80% of product hypotheses, even from smart teams, do not lead to the expected positive metric impact. This high failure rate underscores the importance of fast validation and iterative testing to avoid costly, prolonged development of incorrect solutions.
Product-Led Growth (PLG)
A strategy where the product itself serves as the primary driver for customer acquisition, conversion, and expansion. It requires understanding the unique problems and psyche of self-serve users, often differing from those addressed by traditional sales-led approaches.
Developer User Psyche
Developers are a distinct audience characterized by an aversion to sales interactions and a need to 'prove it to themselves' by building. Their high stakes (job security, team trust) mean they prioritize self-serve experiences and hands-on validation over marketing claims or sales pitches.
8 Questions Answered
Individuals can proactively grow their careers by consistently gathering and sharing valuable customer insights with leadership, building a personal brand as a subject matter expert, and taking initiative to propose solutions to problems they identify, even if it's outside their immediate role.
Yes, in some cases, adding 'good friction' like relevant questions to a signup flow can improve conversion. This happens when the questions address user anxieties or provide reassurance, making the user feel more confident and understood, rather than simply creating a barrier.
Roughly 80% of product hypotheses fail because success requires perfect alignment of many factors: understanding the problem, the customer, the timing, the solution, and its presentation. Any misalignment in these areas can lead to an experiment not achieving its intended impact.
Teams can increase their overall success by embracing the high failure rate and focusing on validating ideas faster through cheaper methods like painted doors or mocks before investing in expensive A/B tests. This allows for more experiments and quicker learning from failures.
Not necessarily. While a 95% confidence interval is critical in fields like pharmaceuticals, growth teams can sometimes accept lower confidence intervals (e.g., 80-90%) to run significantly more experiments. This approach, when combined with strong qualitative validation, can lead to a greater net positive impact over time.
Growth teams should ideally be evaluated over longer periods, such as a year, rather than weekly or monthly. This allows them enough time to run numerous experiments, learn from failures, and achieve meaningful, sustained metric movement, rather than being pressured into vanity metrics or data manipulation.
Companies should 'reset' their thinking for PLG by starting with the unique problems and psyche of self-serve users, rather than simply chopping up an existing enterprise offering. The product should be designed to 'sell' by addressing these specific user needs and building confidence.
Developers are a distinct audience who often skip marketing websites to go directly to signup flows, have a strong aversion to talking to sales, and need to 'prove it to themselves' by building a proof of concept. This stems from the high stakes involved in their role, where product failure is their responsibility.
16 Actionable Insights
1. Carve Your Own Career Path
Proactively seek out and share valuable customer insights, as executives often lack direct customer contact, making your observations a powerful way to build influence and create new opportunities for yourself, rather than waiting for a path to be carved.
2. Proactively Share Customer Insights
Start sharing a ‘voice of the customer’ report or similar insights proactively, even if not asked, to establish yourself as an expert and build trust with senior leaders, which can lead to new initiatives and roles.
3. Ungate Your Knowledge
Share your unique skills and insights beyond your explicit role (e.g., communication tips, customer knowledge) to build your personal brand as a subject matter expert (SME) and open new opportunities within the company.
4. Frame Proposals as Manager Support
When proposing new initiatives or career paths, frame them as supporting your manager and the company, rather than working against them, as this can accelerate your growth and aid your manager in advocating for you during promotion discussions.
5. Prioritize Iterative Experimentation
Opt for iterative experimentation over full redesigns, as most hypotheses fail (80-90% of the time), allowing for faster learning and adaptation rather than investing heavily in potentially flawed large-scale changes.
6. Embrace Failure as a Compass
View experiment failures not as roadblocks but as a compass guiding you to the right solutions, as the high failure rate of hypotheses means rapid iteration and learning from what doesn’t work is key to eventual success.
7. Validate Hypotheses Quickly
Before investing in expensive A/B testing, use cheaper, faster validation methods like ‘painted doors’ or mocks to quickly invalidate hypotheses and reduce the overall failure rate of more resource-intensive experiments.
8. Adjust Experiment Confidence Levels
Consider using lower confidence intervals (e.g., below 95%) for experiments in non-critical areas to significantly increase the number of experiments run annually, leading to a net positive impact over time, especially given the high failure rate of individual hypotheses.
9. Corroborate Risky Experiment Results
When accepting lower confidence intervals in experiments, always harden your validation by corroborating quantitative data with strong qualitative feedback to confirm the underlying hypothesis and reduce the risk of false positives.
10. Set Long-Term Growth Expectations
Educate stakeholders that growth team success cannot be measured on short weekly or monthly timelines due to the inherent high experiment failure rate; instead, commit to annual goals to foster a healthier, more effective environment for true metric movement.
11. Avoid Data Manipulation for Progress
Resist the pressure to make data fit a desired outcome or massage results to show progress, as this leads to vanity metrics and undermines the true purpose of experimentation, which is to validate opportunities and learn.
12. Understand User Psyche in Onboarding
Recognize that ‘good friction’ can improve conversion by addressing user anxieties and providing comfort, such as asking relevant questions in a signup flow to reassure users they are in the right place and capable of using the product.
13. Bury Unpleasant Steps
Embed intimidating or unfamiliar steps (the ‘pill’) within a more comfortable and familiar context (the ‘hot dog’) to reduce user anxiety and improve conversion in self-serve experiences, rather than presenting them as the first step.
14. Create Self-Serve Demos for Non-Developers
Develop ‘create your own demo’ experiences or low-code options for non-developer users to build confidence and excitement, especially when targeting buyers who may find traditional developer tools intimidating but still need to experience the product.
15. Anchor PLG in Customer Problems
When shifting to Product-Led Growth (PLG), do not start by chopping up existing sales-led products; instead, reset and deeply understand the unique problems and psyche of self-serve users to build an experience tailored to their specific needs.
16. Invest in Developer Self-Serve
Recognize that developers are a unique audience who avoid sales and must prove product efficacy to themselves due to high stakes; therefore, heavily invest in robust self-serve experiences to enable their proof-of-concept building and adoption.
7 Key Quotes
I'm not kidding, an improved conversion. Like, there's no personalization, nothing past it, just the questions. An improved conversion by like 5%. Like, just improved sign-ups.
Laura Schaffer
Ultimately the learning here was bad friction is bad and good friction is good, right? There's no such thing as being simple as just all friction is bad.
Laura Schaffer
If it's not embarrassing, you've gone too far.
Laura Schaffer
Failure doesn't have to be a wall. It can be a compass, right? It can be the thing that leads you to the right thing.
Laura Schaffer
You can literally, if you just run 10 in a year, odds are maybe two random impact, two of a course of an entire year if you kind of take that approach.
Laura Schaffer
What it's missing, I think is, and a lot of times it's easy to miss is that when we're doing PLG and we're shifting from sales to PLG, we need to reset. We need to recognize that, you know, again, this is, this is sales, sales via the product.
Laura Schaffer
Developers, like almost two of one, do not look at your marketing website at all. They go straight to your signup flow.
Laura Schaffer
3 Protocols
Proactive Career Growth Framework
Laura Schaffer- Do your current job well and keep notes of successes.
- Actively get to know your customers and their pain points, not just in your specific area but broadly.
- Pull in customer insights and bring them to life by sharing them with senior leaders and across the company (e.g., via a 'Voice of the Customer' report or sessions).
- Build your brand as a subject matter expert (SME) in customer understanding and other valuable skills.
- Propose new ideas or initiatives based on these insights, leveraging the trust and recognition you've built.
Validating Hypotheses Faster (Pre-A/B Test)
Laura Schaffer- Use 'painted doors' to test the concept and idea of a feature before it exists.
- Create mocks or prototypes of the experience and put them in front of users to observe engagement and invalidate hypotheses.
- Only move to expensive A/B testing for ideas that have been vetted and show promise through these faster, cheaper methods.
Addressing User Psyche in Onboarding (Pill in the Hot Dog Analogy)
Laura Schaffer- Identify potentially 'scary' or intimidating steps for new users (e.g., complex technical concepts, unfamiliar processes).
- Instead of making these steps prominent, 'bury' them within a more comfortable or familiar context.
- Present the user with something they are comfortable with first (e.g., code samples for developers) to build confidence.
- Embed the 'scary' step within this comfortable context, making it less intimidating and easier for the user to consume.
- Validate this approach with experiments, even if it means unconventional solutions like temporarily redirecting users to different pages.