The role of AI in product development | Ryan J. Salva (VP of Product at GitHub, Copilot)
Ryan Salva, VP of Product at GitHub, discusses the incubation and launch of GitHub Copilot, an AI-powered code auto-completion tool. He shares insights on fostering innovation within large companies, managing R&D teams, and navigating the ethical and operational challenges of bringing AI products to market.
Deep Dive Analysis
12 Topic Outline
Ryan Salva's unique background and career path
Transitioning from Microsoft to GitHub's product leadership
What GitHub Copilot is and how it works
Educational applications of GitHub Copilot
The origin story of GitHub Copilot and OpenAI collaboration
Developing Copilot's user experience and performance
Structuring R&D teams for 'moonshot' projects
Transitioning R&D projects to operational product teams
Addressing ethical challenges and responsible AI development
The future of AI in software development
Challenges in scaling GitHub Copilot
Allocating resources for bold bets vs. incremental progress
5 Key Concepts
Arctic Code Vault
A GitHub initiative to preserve a snapshot of public code on silver film for thousands of years in a vault in Finland, similar to a seed vault for flora. This data snapshot was later used to train OpenAI's models for Copilot.
Large Language Models (LLMs)
AI models, like GPT-3 and its derivative Codex, that are trained on vast amounts of text data. GitHub realized that programming languages like Python and JavaScript are also 'languages' that could be used to train these models for code generation.
AI Pair Programmer
The framing of GitHub Copilot as an AI assistant that works alongside a developer, similar to human pair programming. This persona helps guide the AI's intended behavior, ensuring it supports and augments the developer without replacing human decision-making or introducing distractions.
Prompt Crafting
The process of experimenting with different ways to feed input to an AI model to elicit a useful and desired response. For Copilot, this involved figuring out how to prompt the Codex model to return relevant and helpful code suggestions.
Project Horizons
A framework for categorizing projects based on their time horizon, ambiguity, and confidence level. Horizon 1 typically refers to the next year, Horizon 2 to the next three years, and Horizon 3 to the next five years, often associated with 'moonshot' projects.
6 Questions Answered
GitHub Copilot is an AI-powered multi-line autocomplete tool for developers, leveraging OpenAI's Codex model to suggest code and entire functions in real-time within the editor, based on context like variables, class names, and comments.
Copilot originated from a collaboration between Microsoft and OpenAI, realizing that large language models could be trained on programming languages. OpenAI's extensive cloning of GitHub's public repositories for data led GitHub to provide a curated snapshot (from the Arctic Code Vault) for training, which then enabled the development of code-generating AI.
GitHub ring-fences a dedicated team called GitHub Next for second and third horizon projects (moonshots), keeping them separate from operational product teams. This allows them space to create and experiment without immediate pressure for monetization or adherence to strict product fundamentals.
Key considerations include ensuring continuity of expertise by moving researchers temporarily to the new product squad for knowledge transfer, basing researcher return on actual replacement in seat rather than calendar dates, giving the product team full ownership of the roadmap, and managing the cultural shift as engineering fundamentals become critical for an operational product.
GitHub addresses ethical concerns through extensive dialogue with legal, privacy, security teams, and the developer community. They frame Copilot as an 'AI pair programmer' to guide its behavior, implement content filtering (now using advanced AI models from Azure Responsible AI), and emphasize that Copilot augments developers, rather than replacing them or existing quality assurance processes.
The vision is for AI to infuse the entire development stack, taking away drudgery (like summarizing pull requests or remembering syntax) so developers can focus on creative acts, design patterns, and higher-level problem-solving. It aims to abstract away complexities, making development more accessible and allowing experienced developers to tackle larger challenges.
34 Actionable Insights
1. Allocate R&D Capacity
Reserve 5-10% of your team’s capacity for bold, experimental research projects to explore highly uncertain but potentially transformative bets. This ensures continuous innovation beyond incremental improvements.
2. Balance Product Development
Dedicate approximately 25-30% of team capacity to operations for maintaining in-market products and meeting customer expectations, while reserving ~60% for incremental improvements that realize payoff from past larger bets. This balanced approach ensures stability and continuous growth.
3. Startups: Go All In
For startups, be ‘all in’ on a single big bet, adjusting resource allocation significantly. This concentrated focus is crucial for pursuing a high-risk, high-reward ’lottery ticket’ idea.
4. Ring-Fence R&D Teams
Create separate, ring-fenced R&D teams for ‘moonshot’ projects, keeping them distinct from core product development. This provides the necessary space for experimentation without the immediate pressures of operational products.
5. Empower R&D Experimentation
Give R&D teams the freedom to experiment without immediate pressure for monetization or adherence to core product fundamentals like security, privacy, or uptime. This fosters creativity and allows for novel discoveries.
6. Validate R&D Prototypes
Begin testing R&D prototypes with more customers once a clear problem, a novel solution, and a medium-confidence signal of customer value emerge. This helps validate the idea before significant investment.
7. Transition R&D with Researchers
When transitioning an R&D project to a product team, temporarily embed the original researchers into the new squad for a finite period. This facilitates crucial knowledge transfer and seeds the operational team with foundational expertise.
8. Base Researcher Transition
Ensure researchers return to R&D only after a replacement is fully in place, has picked up all necessary skills, and can effectively do the job. This guarantees continuity of expertise and smooth project handover, rather than adhering to arbitrary calendar dates.
9. Grant Product Team Ownership
Empower the product team taking over an R&D project with full ownership and control of the roadmap. This ensures they feel responsible for the product’s future and can respond directly to customer feedback.
10. Integrate Engineering Fundamentals
Introduce engineering fundamentals and cultural change management when transitioning R&D projects to operational product teams. This helps researchers adapt to the demands of maintaining a service and ensures a good mix of skills for long-term product health.
11. Define Horizons by Ambiguity
Define project horizons (e.g., Horizon 1, 2, 3) based on the measure of ambiguity and confidence level, rather than strictly by calendar dates. This allows for a more flexible and realistic approach to long-term planning.
12. Create AI Tool Persona
Develop a persona or framing for AI tools, such as an ‘AI pair programmer,’ to define appropriate behavior and guide product principles. This helps clarify expectations and manage user interactions.
13. Engage Diverse AI Stakeholders
Involve a wide range of stakeholders, including legal, privacy, security champions, and end-users, in conversations about ethical AI product development. This comprehensive approach helps navigate complex challenges and gather varied perspectives.
14. Use Advanced AI Moderation
Leverage advanced AI models for content moderation and sentiment detection, moving beyond crude or simple blocklists. This allows for more nuanced and context-aware filtering of potentially offensive content.
15. Position AI as Augmentation
Clearly position AI tools as augmentation for human work, not as replacements for developers or existing quality assurance processes like static analysis and unit tests. This manages expectations and encourages responsible integration.
16. Foster AI Community Dialogue
Actively foster community dialogue and ownership for AI tools, especially those trained on public data. This transparency and engagement help address concerns and build trust with users.
17. Maintain Healthy AI Skepticism
Cultivate and maintain a healthy skepticism towards any AI tool due to its potential for both significant benefit and harm. This critical mindset encourages accountability and responsible development.
18. Address AI Anxieties
Proactively address user anxieties and skepticism about AI through guided conversations and dialogue. This helps the community understand the technology and its implications.
19. Master AI Prompt Crafting
Invest time in mastering prompt crafting for AI models to ensure they return useful and relevant responses. The quality of output heavily depends on how effectively the model is prompted.
20. Optimize Developer Tool Latency
Optimize developer tools for low latency, aiming for around 200 milliseconds for suggestions or responses. This is crucial for helping developers stay in their ‘flow state’ without interruption.
21. Align Career Motivations
Align your career path with your core motivations, such as helping people create. This personal alignment can lead to greater impact and job satisfaction.
22. Focus on Future Potential
When making significant career or product decisions, focus on the future potential of an opportunity rather than just its current state. This forward-looking perspective can unlock greater growth.
23. Go Where Innovation Thrives
Seek out opportunities and environments where energy, focus, and innovation are concentrated. This strategic move can maximize your impact and exposure to cutting-edge developments.
24. Spend Time with Customers
As a product manager, dedicate significant time to engaging with customers and potential customers. This direct interaction is essential for understanding their needs, concerns, and feedback.
25. Use AI for Faster Learning
Leverage AI tools to accelerate learning new codebases and programming languages. AI can help build mental maps and provide context, making the learning process more efficient.
26. Automate Drudgery with AI
Explore using AI to automate repetitive and tedious tasks, such as summarizing pull requests. This frees up developers to focus on more creative and impactful work.
27. Focus on Creative Problem-Solving
Utilize AI tools to offload rote tasks, allowing you to concentrate on higher-level activities like designing patterns, enhancing user experience, and creative problem-solving. This elevates the quality of your work.
28. Lower Entry Barrier with AI
Leverage AI to lower the barrier to entry for new developers and enable experienced developers to tackle larger, more complex problems. This broadens participation and pushes the boundaries of innovation.
29. Try GitHub Copilot
Take advantage of GitHub Copilot’s 60-day free trial to experience AI-powered code auto-completion. Provide feedback on your experience to help improve the product.
30. Optimize Analytics with Amplitude
Try Amplitude for free to optimize your analytics stack, gain self-service product data, and drive conversions, engagement, growth, and revenue. This helps you understand users and make data-driven decisions.
31. Read UX & Character Books
Read ‘Make It So’ for inspiration on user experience design from sci-fi, and ‘Brief Interviews with Hideous Men’ for unique character insights. These books offer diverse perspectives for personal and professional growth.
32. Listen to Memory Palace
Listen to ‘The Memory Palace’ podcast for engaging, 20-minute storytelling vignettes, primarily focused on American history. This can provide unique insights and entertainment.
33. Watch Arrival Movie
Watch the movie ‘Arrival,’ which, while ostensibly about aliens, is deeply compelling in its exploration of language and memory. This offers a thought-provoking viewing experience.
34. Interview Question: Teach Me
Use the interview question ‘Teach me something new in one minute’ to assess a candidate’s completeness, complexity, and clarity in conveying information. This reveals their communication and teaching skills.
5 Key Quotes
Copilot is essentially that magnified by many lines of code. It is multi-lined autocomplete that is fundamentally powered by an AI model called Codex, which is a derivative of another kind of one that you might be familiar with, GPT-3.
Ryan Salva
Copilot helps developers stay in the flow by bringing all of that information into the editor, preventing them from having to go check out documentation, or watch a tutorial, or go to Stack Overflow, and either find an answer, or worse, have to ask a question and wait for an answer.
Ryan Salva
Language models aren't just English and Spanish and German and Korean and Japanese, right? But Python and JavaScript and Java and C Sharp and Clojure, all of these are languages too.
Ryan Salva
We do not want co-pilot auto-generating code where a thinking, reasoning, breathing human being is not on the other side of that keyboard making reasoned decisions.
Ryan Salva
AI is an enabler for developers to focus on the creative work, to stay in the flow, to be able to move faster.
Ryan Salva