Building a world-class data org | Jessica Lachs (VP of Analytics and Data Science at DoorDash)
Jessica Lacks, VP of Analytics and Data Science at DoorDash, shares insights on building impactful data teams, advocating for a centralized model, hiring for curiosity, and selecting effective metrics to drive business outcomes.
Deep Dive Analysis
14 Topic Outline
Centralized vs. Embedded Analytics Teams
Benefits of a Centralized Analytics Team
Balancing Proactive and Reactive Work
Advice on Pushing Back Effectively
Hiring for Curiosity and Problem Solving
Succeeding with a Non-Traditional Background
Early Days and Culture at DoorDash
Encouraging Cross-Functional Roles and Extreme Ownership
Defining Effective Metrics and Aligning Incentives
Simplifying Metrics for Better Outcomes
Focusing on Edge Cases and Fail States in Metrics
Managing a Global Data Organization
Leveraging AI for Analytics Team Productivity
Building Diverse and Skilled Data Teams
4 Key Concepts
Centralized Analytics Model
An organizational structure where analytics professionals report up through a central data organization rather than being embedded within specific business units. This model aims to maintain a consistent talent bar, offer growth opportunities, ensure consistent methodologies, and foster a strong team culture, while still having team members work closely with partner teams.
Extreme Ownership (in Data)
The expectation for data professionals to take full responsibility for outcomes, going beyond their traditional role definitions to solve problems. This can involve tasks like calling customers for qualitative research or performing operational duties if needed to achieve business goals.
Proxy Metrics
Short-term, measurable metrics that are used to predict or drive long-term business outcomes. The goal is to find inputs that can be quickly experimented with and iterated upon, which are proven to influence a desired long-term output like retention.
Edge Cases and Fail States (in Metrics)
Specific, often rare, scenarios or negative outcomes (e.g., 'never delivered' orders) that, despite their low frequency, have a disproportionately high negative impact on customer experience, brand, and cost. Focusing on these can reveal significant opportunities for improvement that average metrics might obscure.
11 Questions Answered
A centralized analytics model, where data professionals report up through a central organization but are embedded in cross-functional pods, is superior for driving business impact and fostering talent.
Teams must be intentional about carving out time for exploratory work and deep dives, setting goals around finding insights through self-directed work, and using hackathons to dedicate time to these activities.
Establish a culture where leadership sets the operating model, align goals with business partners, and always share the tradeoffs of taking on new work, making it a conversation about prioritization.
Beyond technical skills, look for curiosity and self-motivation to dig deeper without being told, and the ability to problem-solve under ambiguity and pivot with new information.
Focus on solving problems pragmatically, be willing to learn new skills out of necessity, and prioritize driving business impact, leveraging the technical skills of the team you build.
This culture comes from the top, with leaders exhibiting and expecting extreme ownership over outcomes, encouraging team members to go beyond their defined roles to solve problems, even if it means doing non-traditional tasks like calling customers.
Focus on simple, understandable metrics that drive long-term outputs, even if they are proxy metrics. Quantify all business levers in a common currency (e.g., GOV or volume) to enable informed trade-offs across teams.
Rare, negative experiences (e.g., 'never delivered' orders) can have a disproportionately high impact on customer churn and cost. Quantifying their impact and setting goals to eliminate them can reveal significant opportunities often missed by focusing only on averages.
While there are complexities like different currencies, languages, and regulations, many problems and human behaviors are surprisingly similar across countries, allowing for the application of lessons learned from one market to another.
AI tools can empower non-technical users to perform tasks like editing SQL queries independently, reducing the burden on the analytics team and freeing them up for higher-impact work.
Actively recruit individuals from non-traditional backgrounds and different functional areas (e.g., engineering, ops, finance) who have acquired technical skills on the job. Also, seek candidates with experience from different company sizes and stages to bring diverse perspectives.
25 Actionable Insights
1. Adopt Centralized Data Org
Structure your analytics team as a central model, rather than embedding them in business units, to ensure a consistent high talent bar, clear growth opportunities, standardized methodologies, and a strong team culture.
2. Drive Business Impact with Analytics
Position analytics as a business impact driving function, not merely a service. This means finding opportunities, having a point of view on decisions, and answering ‘so what do we do now’ instead of just ‘why’.
3. Use Short-Term Proxy Metrics
Identify and measure short-term proxy metrics that reliably drive desired long-term outcomes. Long-term metrics like retention are often too slow to meaningfully impact in the short term, hindering quick experimentation and iteration.
4. Keep Metrics Simple and Understandable
Choose simple, intuitive metrics that are easily understood and discussed across the company, even if not perfectly comprehensive. These are more effective at driving real outcomes than complex, composite scores that nobody understands.
5. Quantify Levers in Common Currency
Quantify all business levers (e.g., price, delivery times) in terms of a common currency like Gross Order Value (GOV) or volume. This allows for quick, informed tradeoff decisions across different teams and initiatives.
6. Target Edge Cases and Fail States
Actively seek out and set concrete goals around eliminating rare but costly edge cases and fail states (e.g., ’never delivered’ orders). These disproportionately lead to churn and expense, and are often missed by focusing only on average metrics.
7. Cultivate Extreme Ownership
Instill a culture of extreme ownership over outcomes, encouraging team members to go beyond their defined roles (e.g., data scientists calling customers) to solve problems and achieve goals, doing whatever is needed to win.
8. Intentionally Carve Out Exploratory Time
Be intentional about carving out time for exploratory work and deep dives, setting goals for self-directed insights (e.g., hackathons). This prevents valuable long-term opportunity-finding from being overshadowed by immediate inbound requests.
9. Communicate Tradeoffs for Prioritization
When faced with multiple tasks, communicate the tradeoffs to business partners. Ask if new requests are more important than existing priorities, making them aware that adding new work means something else must drop.
10. Hire for Curiosity and Self-Motivation
Prioritize hiring for curiosity and self-motivation, as these traits drive individuals to proactively investigate anomalies and opportunities without being explicitly told, adding significant value to the team.
11. Assess Reaction to Being Wrong
During hiring, observe how candidates react to being told they are wrong or presented with new information. Their ability to pivot, take new context, and make decisions with incomplete data is a crucial soft skill.
12. Develop a Point of View
Cultivate the ability to form a point of view and make decisions even when presented with incomplete information. This reflects real-world scenarios where perfect data is rarely available, and a direction must be chosen.
13. Identify Missing Data Gaps
Proactively consider what data might be missing or unobservable (e.g., login failures where users don’t enter the system). These gaps can hide critical problems and opportunities for improvement that won’t appear in standard metrics.
14. Solve Problems from First Principles
Approach problem-solving by focusing on immediate needs and unblocking yourself using first principles. This helps avoid getting overwhelmed by the larger, long-term vision and enables organic learning and growth.
15. Hire for Diverse Backgrounds
Actively seek to hire individuals from diverse professional and educational backgrounds into your data team. This creates a unique environment where varied expertise and perspectives can foster mutual learning and growth.
16. Continuously Re-evaluate Prioritization
Maintain good hygiene by having ongoing conversations with business partners to constantly re-evaluate and align on team priorities, ensuring work remains focused on the most impactful tasks.
17. Build Goodwill with Quick Asks
Occasionally fulfill quick, easy requests from partners, even if not top priority, to build goodwill and strengthen relationships, especially when it takes minimal effort.
18. Empower Non-Technical Users with AI
Leverage AI tools (e.g., chatbots for SQL query editing) to empower non-technical users to self-serve data needs. This reduces the analytics team’s bandwidth burden and increases overall company productivity.
19. Embrace a Customer-First Culture
Cultivate a ‘customer-first’ culture that extends to all stakeholders—consumers, dashers, and merchants—to ensure empathy and focus on their experiences, which is critical for business success.
20. Implement WeDash Program
Institute a program like ‘WeDash,’ where all employees regularly engage in customer-facing roles (e.g., dashing, customer support) to build empathy with all user groups and identify product bugs.
21. Learn from Bad Metrics
Recognize that significant learning about effective metric selection often comes from the experience of choosing and working with ineffective or ‘bad’ metrics.
22. Practice Truth Seeking
Actively practice truth-seeking by diligently discerning fact from fiction amidst widespread misinformation, recognizing this as a critical individual responsibility and company value.
23. Sleep on Difficult Problems
When facing a difficult problem or crafting a response to a tense issue, allow yourself to sleep on it, as a fresh perspective in the morning often leads to better solutions and clearer communication.
24. Use Libby App for Reading
Utilize the Libby app to access and enjoy books from your local public library, supporting community resources and personal reading habits.
25. Try Korean Sunscreens
Experiment with Korean sunscreens, such as those from Beauty of Joseon or Isntree, as they are noted for their superior quality and pleasant wear compared to U.S. alternatives.
7 Key Quotes
For me, analytics is a business impact driving function and not purely a service function.
Jessica Lachs
Not just answering the why, but answering the so what. So what do we do now that we know this?
Jessica Lachs
Retention is a terrible thing to goal on because it's almost impossible to drive in a meaningful way in the short term.
Jessica Lachs
Ultimately, you want to find a short term metric you can measure that drives a long term output.
Jessica Lachs
Yes, you are a data scientist, but your goal is to figure out what's happening. And if that means that you're going to pick up the phone and call customers, then that is what you're going to do.
Jessica Lachs
I sort of joke that I have a job I'd never be hired for because I don't have a traditional data science background.
Jessica Lachs
Sleep can solve lots of problems.
Jessica Lachs
4 Protocols
DoorDash's WeDash Program (Employee Empathy & Product Improvement)
Jessica Lachs- Four times a year, all employees go out and go dashing or do customer support.
- This helps employees use the product and build empathy with consumers, dashers, and merchants.
- It also helps catch bugs in the product.
Effective Pushback on Data Requests (Prioritization & Communication)
Jessica Lachs- Establish a culture where leadership sets the rules of working and the operating model, so junior folks aren't always forced to say no.
- Align team goals with business partners' goals.
- When a new request comes in, share the tradeoffs by asking if the new ask is more important than other planned work.
- Have a conversation with business partners to re-evaluate prioritization, making sure the team works on the most impactful things.
- Occasionally, quickly fulfill small, easy requests to build goodwill.
Hiring for Curiosity (Interview Technique)
Jessica Lachs- Include something 'not quite right' within a business case presented to the candidate.
- Observe if the candidate notices the discrepancy.
- If not, point it out and see how they react to being told they're wrong, how they take new information, and pivot.
- Push candidates to make a decision or have a point of view even without full information.
Defining Effective Metrics (Jessica Lachs' Principles)
Jessica Lachs- Find a short-term proxy metric that can be measured and drives a long-term output (e.g., not directly goal on retention, but on inputs that drive retention).
- Keep metrics simple and intuitive, even if a composite metric might seem 'more perfect.' People need to understand and have an intuition around the metric.
- Quantify all business levers (e.g., price, selection, quality) in terms of a common currency (e.g., GOV, volume) to enable informed trade-offs across different teams and initiatives.
- Actively look for and set concrete goals around eliminating edge cases and fail states (e.g., 'never delivered' orders) as they have a disproportionately high negative impact despite low frequency.