The future of AI in software development | Inbal Shani (CPO of GitHub)
Inbal Shani, Chief Product Officer at GitHub, discusses the future of software development with AI, focusing on Copilot's impact, common mistakes companies make, and the importance of developer happiness and testing. She also shares insights on GitHub's design philosophy and leadership.
Deep Dive Analysis
13 Topic Outline
Generative AI's Overhyped and Underhyped Aspects
The Future of Software Development with AI
GitHub Copilot Usage and Efficiency Statistics
Common Mistakes When Adopting AI Workflows
GitHub's 'Dogfooding' Approach to Product Development
Design Philosophy and Success Metrics for Copilot
AI as a Collaboration Tool and Potential Losses
Retrospective on the Generative AI Boom
Inbal Shani's Prediction for the Future of AI
Fostering Innovation and Experimentation at GitHub
The Role and Success of the GitHub Next Team
Advice for Aspiring Product Leaders
Biggest Career Learning: Leading Change Effectively
5 Key Concepts
AI-driven testing
This refers to leveraging AI to generate a comprehensive suite of tests, including unit, integration, load, security, and penetration testing. It's considered underhyped, despite its critical role in validating the increased volume of code generated by AI tools.
Dogfooding
This is GitHub's practice of being the first to try every feature and capability they develop internally. The company runs GitHub on GitHub, ensuring that if their own teams cannot use the tools effectively, their customers won't be able to either, before anything is shipped.
Working Backwards Concept
This product development approach involves starting with the customer's problem or need and then designing the solution. For Copilot, it meant putting themselves in the developer's shoes to create a seamless, intuitive experience without added friction.
Time to Value
This metric measures the duration from when a developer is assigned a task until the full potential or value of that task is realized. It can be defined by factors like revenue generation, product adoption, or time to market, and is a key business-side measurement for AI tool investment.
GitHub Next Team
This is GitHub's research team, composed of applied and research scientists, focused on a three-to-five-year horizon for software development. Their job is to invent the future through papers, experiments, and Proof of Concepts (POCs), maintaining strong synergy with product and engineering teams to make innovations real.
7 Questions Answered
No, generative AI is not expected to replace humans in the near future because AI cannot replicate innovation or creative thinking, and human interaction is essential for generating the data that AI tools rely on.
AI-driven testing is significantly underhyped. As AI generates more code, the importance of AI-powered testing across all types (unit, integration, security, load) becomes critical for validating software quality and security.
Developers will adopt a more systems-thinking and architectural mindset, spending less time on basic code writing and more on understanding connected systems. Hardware development will also improve to meet AI's resource demands, and AI integration will become a standard skill for all developers.
Companies often expect magical changes without proper change management or try to implement AI without first identifying a specific customer problem it can solve. The focus should be on working backward from a problem to see how AI can be leveraged, rather than just using AI because it's hyped.
Copilot's design philosophy is centered on the 'working backwards' concept, placing the developer's needs first. It aims for seamless, intuitive integration with minimal friction, ensuring developers want to use the tool without needing to learn complex new workflows.
GitHub uses a combination of metrics, as there's no single 'right' one. These include code quality improvements, security enhancements (e.g., preventing secrets leaks), developer productivity gains, and ultimately, developer happiness, with 'time to value' being a key business-side metric.
A major learning is the importance of slowing down and explaining the 'why' behind changes. New leaders often move too quickly without building understanding or taking the team along on the journey, which can lead to resistance and lack of support.
25 Actionable Insights
1. Junior Devs: Learn Systems Early
If you’re a junior developer using AI assistants for code writing, dedicate more time from the start to understanding the overall system, environment, and product you’re building. This allows for a deeper understanding that was previously hindered by the need to learn basic coding.
2. Problem-First AI Adoption
Approach AI adoption by first identifying a customer problem and then determining how AI can best solve it, rather than seeking ways to simply ‘do something with AI.’ This ensures AI is used effectively to address real needs.
3. Leverage AI for Comprehensive Testing
Actively explore and implement AI-driven testing across all types (unit, integration, load, security, performance) in software development. This is critical because AI will generate more code, making thorough validation essential.
4. Retain Staff with AI Tools
Do not reduce engineering staff when adopting AI tools like Copilot, as a human must remain in the loop. Instead, use AI to give developers time back, fostering productivity, happiness, and innovation.
5. Boost Dev Happiness & Innovation
Leverage AI to free up developer time, even just 30 minutes daily, allowing them to write more code, prevent burnout, and dedicate more time to collaboration and creative thinking. This sparks innovation and increases retention.
6. Implement AI Change Management
When adopting AI, invest time in a structured change management process across the company, rather than expecting magical adoption. This ensures successful integration and buy-in from all teams.
7. Design AI for Intuitive Use
Design AI tools with a developer-centric philosophy, prioritizing seamless, intuitive integration and minimal friction. This ensures developers will readily adopt and successfully use the tools without added complexity.
8. Measure AI Impact by Time-to-Value
When evaluating AI tools, focus on ’time to value’ (how long until full potential/value is realized) rather than just time saved. This connects AI’s impact on developer productivity and happiness to business outcomes like revenue, adoption, or time to market.
9. Normalize AI Integration
Developers should proactively learn how to integrate and use AI tools as a standard part of their workflow. This will become a customary skill, shifting thinking from early on in their development.
10. Optimize Hardware for AI Scale
Focus on improving hardware (CPUs, GPUs, compute) and optimizing code to meet the resource demands and scale requirements of AI. This is crucial for supporting the growing use of AI in software development.
11. AI as a Collaboration Tool
Leverage AI to improve collaboration and clarify communication, especially when articulating ideas or design sketches. This can significantly shorten feedback cycles and align diverse teams by acting as a universal translator.
12. Personalize AI Tool Adoption
Customize your use of AI tools to align with your personal preferences, using AI for tasks you enjoy less (e.g., writing tests) while retaining manual control over tasks you find fulfilling (e.g., specific coding functions). There’s no single ‘right’ way to use AI; adapt it to enhance your happiness and productivity.
13. Plan for Hybrid AI Models
Prepare for a future where specific, niche AI models coexist with generalized generative AI, recognizing that specialized problems (e.g., high-safety systems) will still require custom-trained models due to generative AI’s limitations.
14. Foster Innovation & Experimentation
Dedicate bandwidth and time for teams to innovate and experiment, accepting that not all experiments will succeed. Embrace a ‘ship to learn’ or ‘fail forward’ philosophy to continuously drive progress and new ideas.
15. Gather Customer Innovation Ideas
Actively engage with customers and your community to understand their frustrations, desires, and innovative ideas. This direct feedback loop is crucial for shaping future product development and identifying next-generation use cases.
16. Fund & Pilot Pitched Ideas
Establish a process for teams to pitch new ideas, backed by research, and then strategically fund and allocate resources for Proof of Concepts (POCs) or dedicated V-teams. This provides a structured path for promising innovations to be explored.
17. Bridge Research to Production
Ensure research and innovation teams (like GitHub Next) are tightly integrated with product and engineering, focusing on how ideas can transition to production from day one. This prevents research from becoming isolated and ensures real-world impact.
18. Protect Research from Tactical Work
Safeguard research and innovation teams from being overloaded with immediate, tactical demands that belong to core engineering or product teams. Their role is to invent the future (3-5 year horizon), not solve short-term problems.
19. Internal Dogfooding for Quality
Ensure your internal teams are the first to extensively test every feature and capability being developed. If your own teams cannot successfully use the tools, your customers likely won’t either, ensuring higher quality before shipment.
20. Develop Broad CPO Skills
Aspiring Chief Product Officers should cultivate a broad skillset beyond just product management, including business acumen, go-to-market strategy, sales understanding, and engineering processes. This holistic view is crucial for effectively leading product development.
21. Think System-Wide Solutions
When solving a problem, always consider its connection to the larger system and how it integrates with other components, rather than focusing solely on your immediate task. This ensures a comprehensive and effective solution from the outset.
22. Master Influential Leadership
As a product manager or leader, prioritize developing strong influential skills to guide engineering, revenue, and other teams towards your product vision. Success often hinges on influencing others rather than direct authority.
23. Learn from All Team Levels
Actively seek to learn from leaders above you, peers across your organization, and those you manage, identifying skills to adopt or behaviors to avoid. This broad perspective enriches your leadership toolbox.
24. Communicate ‘Why’ for Change
When driving change, slow down and clearly explain the ‘why’ behind the proposed changes to your team. People need to understand the problem and rationale to be on board, as not everyone embraces change at the same pace or with the same mindset.
25. Embrace Discomfort, Take Risks
Actively seek out discomfort and take calculated risks to stretch yourself and drive future innovation. Staying too comfortable can hinder personal and professional growth.
5 Key Quotes
Copilot is a co-pilot. It's not a pilot. You still need the human in the loop.
Inbal Shani
The overhype that is happening right now is that generative AI will replace humanity. It's going to be the solution for everything. And I think it's a tool.
Inbal Shani
If you don't try, if you don't experiment, you will never innovate. And if you don't fail, you will never learn from your mistakes.
Inbal Shani
If you feel comfortable, it's not a good thing. You always need to feel uncomfortable to stretch yourself to go to the next thing.
Inbal Shani
You pick and choose how you want to use it. There is no global way that is the absolute truth that everyone has to follow.
Inbal Shani