Becoming evidence-guided | Itamar Gilad (Gmail, YouTube, Microsoft)

1. Shift to Evidence-Guided Development

Move your team and organization from opinion-based decision-making to an evidence-guided approach. This involves balancing human judgment with data to supercharge decisions, leading to more successful products and avoiding wasted resources on unvalidated ideas.

2. Challenge Exec Opinions with Data

When founders or senior leaders propose ideas, ask for the evidence supporting them. If necessary, run secret experiments or gather data to objectively challenge opinions, as even influential leaders like Steve Jobs were willing to change their minds with sufficient evidence.

3. Define User-Centric Goals

Establish clear, overarching, user-centric goals for the entire organization, not siloed by department. Use a “North Star metric” to measure value delivered to users and a “top KPI” for business value, then break these down into “metrics trees” to align teams and assess impact.

4. Adopt the GIST Model

Implement the GIST (Goals, Ideas, Steps, Tasks) model as a meta-framework to structure product development. This breaks down the process into manageable parts, ensuring alignment, systematic idea evaluation, simultaneous learning and building, and team engagement.

5. Evaluate Ideas with ICE

Use the ICE (Impact, Confidence, Ease) framework to objectively and transparently evaluate ideas. Estimate Impact on goals, Ease of implementation, and critically, Confidence in those estimates, to avoid opinion battles and prioritize effectively.

6. Quantify Confidence with Meter

Employ the Confidence Meter tool to quantify the strength of evidence supporting your ideas, ranging from low confidence (opinions, themes) to high confidence (various forms of testing and experiments). This helps teams understand the reliability of their impact and ease estimates and guides investment.

7. Invest Proportionally to Confidence

Tie your investment in an idea to its level of confidence. Start with cheap, low-effort methods to gain confidence, then invest more as positive evidence accumulates, and know when to stop testing if the risk is low.

8. Combine Learning and Building

Integrate learning and building simultaneously, rather than treating them as separate phases. Use a spectrum of validation methods, from inexpensive “fake it” tests to more elaborate experiments and staged releases, to validate assumptions early and efficiently.

9. Prioritize Time to Outcomes

When facing uncertainty, shift the focus from how fast you can get features into production to how fast you can achieve desired outcomes. Evidence-guided methods, by prioritizing learning and validating the “right bits,” are ultimately more resource-efficient and faster to impact.

10. Empower Teams with GIST Board

Replace traditional roadmaps with a GIST board for each team, displaying their key results (goals), current ideas, and planned validation steps. This fosters context, ownership, and dynamic decision-making, allowing teams to autonomously pursue outcomes and learn continuously.

11. Use Fake Door Tests

Before building a full product, create a facade (e.g., HTML mock-up) to test user interest and gather evidence. This allows for early validation without significant development cost, as demonstrated with the Gmail tabbed inbox.

12. Continuously Conduct User Research

Don’t wait until you have an idea to start user research; maintain ongoing research efforts. This ensures you have a continuous stream of data from surveys, interviews, and field observations to inform and validate new ideas efficiently.

13. Involve Developers in Discovery

Break down the traditional divide between planning and execution by inviting developers into the discovery process. This allows them to contribute to idea selection and validation steps, leading to greater engagement and more effective outcomes.

14. Start with Biggest Problem

When adopting evidence-guided practices, identify your organization’s most pressing problem (e.g., unclear goals, constant debates, disengaged teams) and start implementing the corresponding GIST layer first. Avoid trying to transform everything at once to prevent fatigue.

15. Use Outcome Roadmaps

Replace traditional release roadmaps (focused on launching specific features by certain dates) with outcome roadmaps. Define what outcomes you want to achieve by when, allowing flexibility in how those outcomes are met and encouraging continuous learning and adaptation.

16. Critically Evaluate Competitor Features

Do not assume a competitor’s feature is a good idea just because they launched it. Critically evaluate it with your own data and testing, as assuming they know what they’re doing can lead to implementing ineffective ideas.

17. Fish Food for Rough Versions

When you have a rough, incomplete, or unpolished version of a product, test it internally with your own team, a practice referred to as “fish food.” This provides early feedback before broader internal or external testing.

18. Dog Food Product Internally

For more complete versions of your product, implement “dog fooding” by having your internal team use the next version of the product. This helps identify bugs and gather practical feedback before external release.

19. Utilize Stage Releases

Even after a product is built, use staged releases, percentage launches, and holdbacks as final opportunities to learn and validate assumptions. This allows for controlled rollout and adjustments based on real-world usage.

20. Strive to Be of Value

Adopt the motto “Strive not to be a success, but to be of value.” This principle encourages focusing on creating genuine worth for others, which can guide both personal and professional endeavors.