Becoming an AI PM | Aman Khan (Arize AI, ex-Spotify, Apple, Cruise)

Nov 14, 2024 Episode Page ↗
Overview

Aman Khan, Director of Product at Arise AI, shares insights on how to break into and thrive as an AI Product Manager, covering different types of AI PMs, leveraging AI tools for productivity, and strategies for success as a long-term Individual Contributor PM.

At a Glance
23 Insights
1h 17m Duration
15 Topics
5 Concepts

Deep Dive Analysis

Introduction to Aman Khan & AI PM Trends

Defining AI Product Manager Roles

Tools for AI-Powered Product Managers

How to Break Into AI Product Management

Creating a Portfolio to Stand Out as an AI PM

Why AI Enhances, Not Replaces, Product Managers

What Separates Top 5% AI Product Managers

Identifying Effective AI Product Ideas

The 'IKEA Effect' and User Control in AI Products

Habits for Long-Term Individual Contributor Success

The Role of Energy in Individual Contributor Success

The 'Wandering vs. Waiting' Mindset for PMs

Using AI Tools to Amplify Signal and Make Decisions

The Importance of Having Fun in Product Management

Lightning Round

AI Platform PMs

These Product Managers build tools and infrastructure specifically for AI engineers, enabling them to develop, deploy, and monitor AI models and applications. An example is an observability and evaluation platform for artificial intelligence.

AI Product PMs

This type of Product Manager focuses on products where the core experience and secret sauce are centered around AI technology. Their role involves packaging cutting-edge AI research and making it consumable for businesses or consumers, like with ChatGPT or NotebookLM.

AI-Powered PMs

These Product Managers leverage AI technology across their entire role to enhance productivity and build better customer experiences. They utilize AI tools to streamline tasks like prototyping, design, and data analysis, without necessarily building the core AI models themselves.

The IKEA Effect (Applied to AI)

This concept suggests that users feel more empowered and satisfied with an AI-powered product when they have some level of control or input into the experience, rather than it being fully automated. It's about finding the right balance where AI makes things easier but still allows for user agency.

Wandering vs. Waiting

This mindset describes the approach to product discovery, especially in nascent fields like AI. 'Wandering' involves actively exploring unknown possibilities, pushing boundaries, and seeking out new signals, while 'waiting' means holding back for clearer directions or established technologies to emerge.

?
What are the different types of AI Product Managers?

There are three main types: AI Platform PMs (building tools for AI engineers), AI Product PMs (where the core product is AI-centered), and AI-Powered PMs (using AI tools to enhance their existing PM role and product experiences).

?
How can aspiring AI PMs break into the field?

Aspiring AI PMs should build a strong foundation in AI/ML fundamentals, let curiosity drive them to explore applications, and create a portfolio of AI-powered prototypes to demonstrate their skills and interests to hiring managers.

?
How can Product Managers use AI tools to be more efficient?

PMs can use AI tools like Cursor, Replit, V0, Midjourney, and Dall-E to quickly build functional prototypes, create mock-ups, generate graphic designs, and even automate parts of data analysis or signal amplification.

?
Why are Product Managers still essential in the age of AI?

PMs are more important than ever because AI tools excel at building things when told what to build, but the hardest part remains knowing what to build, identifying problems, articulating clear requirements to AI, and having the taste to discern what will succeed in the market.

?
What distinguishes top 5% AI Product Managers from typical ones?

Top AI PMs avoid simply replicating existing AI interfaces (like building another chatbot) and instead focus on identifying unique business problems, designing novel AI interfaces, and finding innovative ways to apply AI technology to create seamless user experiences.

?
How can companies find good, AI-oriented product ideas?

Companies can find good AI ideas by measuring the number of AI prototypes or experiments attempted, conducting hackathons to encourage hands-on exploration, and obsessing over user experience details to identify where AI can create truly magical and empowering interactions.

?
What are key habits for long-term individual contributor (IC) success?

Key habits include bringing high energy to interactions, being a 'player-coach' by getting into the details and doing the work alongside the team, embracing a 'wandering' mindset to explore new directions, and leveraging AI tools to amplify signal and scale personal impact.

?
How can PMs make decisions when the path forward is unclear?

PMs should embrace the 'wandering' mindset, feeling comfortable with uncertainty and continuously exploring possibilities until a clear direction emerges, rather than waiting for executives or established technologies to define the path.

?
What is the 'IKEA Effect' and how does it apply to AI product design?

The 'IKEA Effect' in AI product design means that users appreciate having some control or input over an AI-powered experience, even if the AI could fully automate it. Products that leave 'knobs and levers' for users tend to create a more empowering and satisfying experience.

1. Focus on Problem, Not AI

Prioritize deeply understanding and solving customer problems, using AI as a tool to achieve solutions rather than building AI for its own sake. This approach ensures you’re addressing genuine business needs and not just replicating existing AI interfaces.

2. Build AI Product Portfolio

Create a portfolio of AI prototypes or products, even if they are just functional mock-ups, to demonstrate your thinking and building capabilities. This practical experience helps you stand out to hiring managers and showcases your interest in the space.

3. Embrace the “Wanderer” Mindset

As an IC PM, be comfortable with ambiguity and actively explore unknown paths to discover new product directions, continuously iterating until a clear product pull emerges. This is crucial for zero-to-one product development, even when others have clear roadmaps.

4. Bring High Energy to Work

Cultivate a positive and engaged demeanor in meetings and interactions, as bringing high energy can reduce friction, foster positive interactions, and elevate team morale and performance. Actively participate and demonstrate commitment to the work at hand.

5. Scale Yourself with AI Tools

Actively seek out and implement AI tools to enhance your personal productivity and ability to gather and analyze information, effectively scaling your capacity. For example, use LLMs to analyze meeting transcripts for common customer pain points.

6. Design for User Control

When designing AI products, incorporate the ‘IKEA effect’ by leaving users some control or ‘knobs and levers’ over the experience, even if full automation is possible. This increases user engagement and satisfaction by making them feel empowered.

7. Balance Value & Experimentation

Continuously deliver customer value while also creating dedicated space for AI experimentation and learning, accepting that some experiments will fail. This iterative approach is essential for adapting to the fast-evolving AI landscape and driving better deployment.

8. Continuously Learn & Have Fun

Prioritize having fun, continuous learning, and genuine curiosity in your work, as this mindset fosters faster iteration, higher energy, and greater long-term success. This approach makes the journey of product development more enjoyable and effective.

9. Deeply Understand AI Fundamentals

Build a foundational understanding of machine learning and AI concepts, such as how LLMs work, to grasp the technology’s capabilities and boundaries. Then, apply this knowledge to problems and industries that genuinely interest you.

10. Use AI for Prototypes

Leverage coding AI tools like Cursor and Replit to quickly build functional prototypes, enabling you to demonstrate possibilities and tell a story effectively in early product discussions. This raises the resolution of initial ideas.

11. Develop Strong AI Prompting

Cultivate strong prompting skills for AI tools to effectively guide them in generating desired outputs for prototypes, UIs, or graphics. The quality of the output depends on how well you can prompt the AI.

12. Experiment with New AI Tech

Actively experiment with new AI technologies as a consumer to discover ‘aha moments’ and identify potential applications for your product or team. This curiosity drives skill development and product innovation.

13. Be an Internal AI Expert

Position yourself as the internal expert on leveraging AI tools, combining deep customer problem understanding with knowledge of AI capabilities. This enables you to effectively advocate for what should be built within your company.

14. Avoid Replicating Chatbots

Do not blindly replicate popular AI interfaces like chatbots; instead, deeply analyze business needs and customer problems to design unique AI solutions. The most impactful AI interfaces may look very different from conventional chatbots.

15. Critically Evaluate AI Agents

Critically evaluate whether to build AI agents in-house or leverage foundational models and their agentic layers from external providers. Prioritize seamless integration of AI into existing products to create a magical, almost invisible, user experience.

16. Measure AI Experimentation

Establish metrics for AI prototyping and experimentation, such as ’number of shots taken,’ to measure the impact of AI initiatives beyond direct revenue, especially in early stages. This helps track progress when direct business metrics aren’t immediately affected.

17. Organize AI Hackathons

Organize hackathons to encourage hands-on experimentation with AI across the organization, making the technology more approachable and generating diverse problem-solving ideas. Use these to identify suitable problems for AI solutions.

18. Conduct AI Product Teardowns

Regularly conduct internal teardowns or webinars of cutting-edge AI products to foster a deeper understanding of the space and learn from successful implementations. This helps the team understand how AI works in practice.

19. Leverage AI for Empathy/Learning

Proactively ‘get into the details’ by taking on tasks outside your typical PM role, such as customer outreach or even sales, to gather insights and demonstrate commitment. This also builds empathy for team members’ roles and challenges.

20. Network Uniquely (Top 3 Content)

When cold messaging or networking, offer to share your top three favorite books, movies, or podcasts in exchange for theirs. This creates a unique and engaging way to stand out and build meaningful connections.

21. Understand AI PM Types

Familiarize yourself with the three main types of AI PMs (Platform, Product, Powered) to better understand potential career paths and how AI can integrate into or enhance various product management roles.

22. Use AI for UI/Landing Pages

Leverage AI UI generation tools like Vercel’s V0 to create beautiful landing pages or working UIs from prompts. This provides a strong, high-resolution starting point for design discussions.

23. Use AI for Graphic Design

Employ AI image generation tools like MidJourney or DALL-E for graphic design, logo creation, or visual storytelling. This makes user stories more approachable and tangible for stakeholders.

Your time is limited, so don't waste it living someone else's life.

Steve Jobs (quoted by Aman Khan)

If you're in love with the problem, you're trying to push the boundaries of what's possible with technology to solve that problem.

Aman Khan

Being an AI product manager feels like the highest leverage position you can be in at the company right now, especially in the age of AI.

Aman Khan

You have to be able to walk and chew gum.

Kevin Yen (quoted by Aman Khan)

It's just more fun to have, you know, to be the to be like high energy, it's just more fun to, you know, like really care about the thing that you're working on.

Aman Khan

Building a Portfolio to Stand Out as an AI PM

Aman Khan
  1. Start with the foundation of what machine learning and AI are, and what the technology can do.
  2. Let your curiosity drive you to apply this knowledge to problems or industries that interest you.
  3. Build a portfolio of products (even prototypes) using AI tools to demonstrate your thinking and interests.
  4. Use this portfolio to shortcut the hiring process by showing hiring managers you can do the job and are excited about the work.

Finding Good AI-Oriented Product Ideas

Aman Khan
  1. Measure 'Shots on Goal': Track how many AI prototypes or experiments are being attempted, even if they don't directly move business metrics initially.
  2. Conduct Hackathons: Get everyone in the organization hands-on with AI tools to remove aversion and generate ideas. Focus on problems to be solved, not just applying AI.
  3. Obsess About User Experience Details: Study successful AI products to understand what makes them 'magical' and identify where AI provides true value, often by making creation easier or lowering barriers, rather than full automation.

Thriving as an Individual Contributor PM

Aman Khan
  1. Bring Energy: Show up to meetings with enthusiasm and a positive mindset to reduce friction and encourage productive conversations.
  2. Be a 'Player-Coach': Be willing to get into the details and do the work alongside your team, demonstrating commitment and building empathy.
  3. Embrace 'Wandering': Actively explore unknown possibilities and push the boundaries of what the company thinks is possible, rather than 'waiting' for clear directions.
  4. Amplify Signal with AI Tools: Use AI (e.g., LLMs with long context windows, Gong transcripts) to process large amounts of information and extract key insights or customer pain points, scaling your ability to find signal through noise.
  5. Just Have Fun: Let curiosity and enjoyment drive your work, as this fosters faster learning and iteration.
1 hour
Time to build an app using AI tools (e.g., YouTube videos) Refers to how quickly one can build an app with modern AI tools.
45 minutes
Time Greg Eisenberg demonstrates building a product Refers to a format pioneered by Greg Eisenberg for building products with AI tools.
5 minutes
Time to build a working sign-up page prototype on a phone using Replit Aman Khan's personal experience demonstrating the ease of AI prototyping.
3 months
Average engineering time to build in-house integrations According to the 2024 state integration survey mentioned by Lenny Rachitsky.
7 times faster
Speed increase for shipping integrations using Paragon Benefit for B2B SaaS companies using Paragon.
9 out of 10
Ratio of failed AI applications in hackathons Aman Khan's observation from a hackathon where 9 out of 10 AI ideas didn't work as expected.