Inside the expert network training every frontier AI model | Garrett Lord (Handshake CEO)
Garrett Lord, co-founder and CEO of Handshake, discusses their new AI data labeling business, which leverages their vast student network to provide expert-level training data for frontier AI labs. This venture rapidly scaled to $50M ARR in four months, demonstrating how companies can disrupt themselves and capitalize on AI opportunities.
Deep Dive Analysis
14 Topic Outline
Introduction to Handshake and its AI success story
Explaining Data Labeling and Post-Training AI Models
The Shift from Generalist to Expert Data Labeling
AI's Impact on Entry-Level Jobs and Human Productivity
The Continuous Need for Human Experts in AI Advancement
Handshake's Discovery and Launch of Its AI Data Business
Incubating a New Business Unit Within an Established Company
Handshake's Unique Competitive Advantage in Expert Data
Scaling the AI Data Business to Meet Unlimited Demand
Challenges of Building a Startup Inside a Startup
Strategies for Successful Internal Startup Incubation
AI's Role in Reinventing Job Matching and Labor Supply
Future of Data Needs and AI Advancement Bottlenecks
Lightning Round and Entrepreneurial Advice
6 Key Concepts
Pre-training (AI Models)
The initial phase of training an AI model where it 'sucks up' the entire corpus of written human knowledge, including books, videos, and internet content, to learn general knowledge and reasoning.
Post-training (AI Models)
The phase after pre-training where AI models are augmented and improved by collecting high-quality, specialized data across various disciplines (coding, math, law, finance) to enhance specific capabilities.
Data Labeling (Expert)
The process where human experts, such as PhDs in specific fields, interact with AI models to identify flaws, provide correct answers, and outline step-by-step reasoning, thereby creating new, high-quality data to improve model capabilities. This contrasts with earlier 'generalist' labeling.
Trajectory Data
Data collected from a user's entire environment, including screen recordings, mouse movements, and voiceovers, as they solve a problem from start to finish, interacting with multiple tools and services. This helps AI models understand human thought processes.
Rubrics (AI Evaluation)
Structured guidelines or criteria, similar to those used in academic grading, that allow an AI model to act as a judge and auto-evaluate responses in domains where there isn't a single guaranteed correct answer, helping models understand what constitutes a 'good' outcome.
AI Native
A term describing young people who have grown up with and are proficient in leveraging AI tools, giving them a significant advantage in productivity and problem-solving in the modern workforce.
7 Questions Answered
Data labeling involves human experts augmenting and improving AI models by providing high-quality, specialized data, identifying model weaknesses, and offering correct reasoning steps. This is crucial because AI models have largely consumed existing internet data, and post-training with expert-generated data is now the primary driver of capability improvements.
The demand has shifted from generalist labor, which performed basic tasks, to highly specialized experts (e.g., PhDs in STEM fields) because AI models have become so advanced that they require nuanced, expert-level data to improve further.
Experts, like PhDs, identify flaws in model reasoning or ground truth answers, provide correct step-by-step solutions, and create new data in specialized domains. This can involve tasks like narrating their problem-solving processes (trajectory data) or developing rubrics for model evaluation.
No, AI is not eliminating entry-level jobs but rather transforming them by providing 'Iron Man suits' that make junior employees significantly more productive, allowing one person to accomplish tasks that previously required multiple specialists.
Handshake's advantage stems from its proprietary network of 18 million students and alumni, including 500,000 PhDs and 3 million master's students. This gives them unparalleled, low-cost access to a highly engaged, hyper-targetable expert audience, leading to higher conversion and retention rates compared to competitors who rely on expensive recruitment.
Key strategies include dedicating the CEO's focus, creating entirely separate teams (engineering, design, finance, etc.) with distinct responsibilities and reporting structures, operating in a separate physical space, establishing a metrics-oriented cadence, and fostering an entrepreneurial culture with clear ownership and high expectations.
The primary bottleneck is the need for new, high-quality, specialized data. While the type of data will evolve (e.g., CAD files, scientific tool use, multimodal data), humans will continue to be essential in generating this data for the foreseeable future, as synthetic data alone is not expected to dominate.
24 Actionable Insights
1. Identify Unfair Advantages for New AI Ventures
Reflect on existing company assets, networks, or capabilities to identify “unfair advantages” that can be leveraged to build new, significant businesses in the AI era. This encourages internal disruption and innovation.
2. Separate New Ventures from Core Business
Acknowledge the inherent difficulty of incubating new ventures within a mature business, as the zero-to-one approach differs significantly from running an established company. This necessitates distinct operational models and structures.
3. Create Dedicated, Focused New Business Unit
Establish a completely separate organizational structure for a new venture, ensuring all team members have no responsibilities in the existing business and clear Directly Responsible Individuals (DRIs). This fosters intense focus and avoids dilution of effort.
4. Maintain Full Separation for New Business
Ensure full separation of all functional teams (engineering, design, operations, finance) for a new business, with each person having only one job: making the new venture successful. This prevents resource contention and maintains singular focus.
5. Foster Startup Culture for New Ventures
Cultivate a distinct, intense startup culture for new ventures, including separate physical spaces, high work expectations (e.g., 24/7 commitment), and compensation tied to new business hurdles. This attracts entrepreneurial talent and drives rapid growth.
6. Hire Entrepreneurs for New Ventures
Prioritize hiring individuals with entrepreneurial backgrounds or those who have previously started companies for new ventures. These individuals are comfortable with ambiguity and possess the drive needed for early-stage growth.
7. Be Transparent About Startup Chaos
Be upfront and transparent about the chaotic nature and high expectations of a new venture, communicating this clearly in separate all-hands meetings and onboarding processes. This sets realistic expectations and attracts suitable talent.
8. Recruit Top Talent from Core Business
Actively recruit top, entrepreneurial talent from the core business to staff the new venture, even if it means asking them to forgo existing responsibilities. This provides critical expertise and leadership for rapid scaling.
9. Adopt Metrics-Driven Operating Cadence Early
Implement a rigorous, data- and metrics-oriented operating cadence (weekly, monthly, quarterly) from the early stages of a new venture. This provides clarity, accountability, and rapid feedback for decision-making.
10. Cultivate Urgency for Unique Opportunities
Instill a sense of urgency and shared mission within the team, emphasizing that a current opportunity is “once in a lifetime” and requires collective commitment to “meet the moment.” This motivates high performance and execution.
11. Leave Nothing to Chance
To avoid regrets in a fast-moving, high-opportunity environment, commit fully by engaging directly with customers, making late-night pushes, and meticulously verifying data. This ensures maximum effort against unlimited demand.
12. Focus on Meaningful AI Solutions
For aspiring entrepreneurs, especially in AI, focus on building solutions that address meaningful societal problems and genuinely help people, such as improving learning or connecting talent with opportunities. This approach can lead to both impactful and successful ventures.
13. Anticipate AI-Driven Hiring Transformation
Recognize that AI will fundamentally reinvent the job search and hiring process, automating tasks like resume review and cover letter creation. This highlights the need for platforms that can effectively match talent with opportunities using AI.
14. Anticipate Continued Human Role in AI Training
Expect that humans will remain essential in the AI training process for the foreseeable future, as models continuously improve and data types evolve. This suggests a long-term opportunity for human expertise.
15. Leverage Expert Data for AI Models
Understand that advanced AI models require expert-level data, not generalist input, to improve capabilities in specialized domains like STEM, accounting, law, or medicine. This shifts the focus from low-cost labor to specialized knowledge.
16. Prioritize Data Quality, Volume, and Speed
When providing data for AI models, prioritize quality above all else, followed by the ability to generate high volumes of data, and then speed of turnaround. These three factors are critical for model builders to test hypotheses and scale improvements.
17. Build Audience Access for Data Moat
Recognize that access to a proprietary, engaged audience is the primary competitive moat in the human data market. This allows for efficient targeting and acquisition of high-quality experts without high customer acquisition costs.
18. Treat Expert Data Labelers as Valued Professionals
Create a “brandy experience” for expert data labelers, treating them with the respect and work expectations appropriate for their high qualifications (e.g., PhDs). This fosters a community and ensures higher quality contributions compared to generalist labor.
19. Address Model Reasoning Weaknesses
Identify and fix where AI models fail in their reasoning steps or ground truth answers, especially in advanced domains, by providing step-by-step instructions and correct solutions. This helps improve the model’s underlying thought process.
20. Leverage AI Data Work for Skill Development
Engage in AI data creation and labeling as a way to get paid while simultaneously learning and advancing skills in your field of expertise. This offers a unique opportunity for professional growth and research advancement.
21. Embrace AI-Native Skills for Career Advantage
Understand that young people who are “AI native” and skilled at leveraging AI tools gain a significant advantage in the job market, enabling them to be more productive and impactful. This suggests a need for current professionals to also become AI-native.
22. Recognize Expert-Driven AI Improvement
Understand that when AI models fail at advanced tasks, it’s often due to the absence of expert-level data, which is actively being addressed by top minds in specific domains. This highlights the ongoing, iterative, and human-centric process of AI advancement.
23. Focus on Post-Training Data
Recognize that most AI model gains now come from post-training, which involves augmenting and improving the data across specific disciplines or capabilities. This is where high-quality, expert-generated data is crucial.
24. Trust Founder Intuition in New Ventures
In new, ambiguous ventures, rely more on founder intuition, especially when leveraging past experiences in building and scaling. This can guide rapid decision-making where established processes may not exist.
6 Key Quotes
There will never be a time like this. I've never seen anything like it. I doubt I'll ever feel anything like this in business again, where there's unlimited demand.
Garrett Lord
The models have gotten so good that the generalists are no longer needed. What they really need is experts.
Garrett Lord
The only moat in human data is access to an audience.
Garrett Lord
The model of today is the worst model you will ever use.
Lenny (quoting Kevin Weil)
Being like AI native, young people are at a huge advantage.
Garrett Lord
Leave nothing to chance. How do you make sure that three months from now, six months from now, you have like no regrets? Get on the plane to go talk to a customer. Make the late night push. Check the data six times over again. Ship the extra feature that helps.
Garrett Lord