Why experts writing AI evals is creating the fastest-growing companies in history | Brendan Foody (CEO of Mercor)

Sep 18, 2025 1h 6m 24 insights Episode Page ↗
Brendan Foody, CEO & co-founder of Mercore, discusses the "era of evals" in AI, Mercore's hyper-growth, and the creation of new high-skilled jobs for training AI models. He shares insights on building a successful company and adapting to the evolving labor market.
Actionable Insights

1. Master AI for Career Survival

Prioritize becoming highly proficient in using AI tools and technologies, as individuals who effectively leverage AI will be more competitive and successful than those who don’t.

2. Embrace AI for Abundance

Adopt a mindset of abundance, focusing on how AI enables you to achieve significantly more and create new possibilities, rather than resisting it out of fear of job displacement.

3. Take Initiative and Build

Overcome the barrier of inaction by taking the initiative to build products or experiences that customers want, investing time and ambition to scale them up, especially with AI making it easier to build.

4. Integrate AI into Daily Work

Actively learn and integrate AI tools into your daily workflows, regardless of your industry, to enhance your capabilities and productivity.

5. Prioritize Customer Obsession

Focus 100% of early company resources on building exceptional products and customer experiences, allowing word-of-mouth and customer love to drive growth before investing heavily in sales and marketing.

6. Track Market Leading Indicators

In fast-moving markets, prioritize identifying and acting on leading indicators of new demand pockets, especially where wealthy customers are willing to pay for solutions, to ensure you build the best product for flagship customers.

7. Define AI Model Success Metrics

To improve AI models, clearly define what success looks like for the model, creating effective measurement systems (evals) that serve as benchmarks and verifiers in reinforcement learning environments.

8. Leverage Strengths, Not Weaknesses

In management, focus on leveraging individuals’ strengths to maximize their impact rather than excessively trying to improve their weaknesses, recognizing that some areas may never be world-class.

9. Foster Intense, Output-Oriented Culture

Build an intense, output-oriented early-stage culture where team members are deeply bought-in and committed to achieving ambitious goals, focusing on results rather than specific hours worked.

10. Maintain High Hiring Standards

Implement incredibly high hiring standards, prioritizing talent density by seeking out exceptional individuals, including former founders or those with significant achievements, to shape the organizational culture.

11. Cultivate a Can-Do Attitude

Cultivate a “can-do” attitude to set ridiculously ambitious goals, as the company’s trajectory often forms around these high aspirations.

12. Find Easy-to-Sell Customers

Seek out customers who are surprisingly easy to sell to, indicating a significant pain point and potential for growth; balance strong conviction in your vision with openness to how the market evolves and your company fits in.

13. Embrace AI Tools in Assessments

When assessing talent or designing tasks, encourage the use of AI tools (like ChatGPT, Codex, Cursor) rather than prohibiting them, focusing on what individuals can achieve by leveraging these technologies.

14. Pursue Elastic Demand Industries

For career planning, focus on industries with elastic demand (e.g., software development, consulting, operations) where increased productivity from AI will lead to greater demand and output, rather than job displacement.

15. Automate Hiring with AI

For companies, leverage AI to automate the manual matching problems in hiring (resume review, interviews) to access a global, unified labor market and improve efficiency.

16. Focus on Human-AI Gaps

Identify and focus on tasks and domains where humans currently outperform AI, as these areas will continue to require human expertise for evaluation and improvement of models for the foreseeable future.

17. Adopt RLAIF for Scalability

Transition from human feedback (RLHF) to AI feedback (RLAIF) by having humans define scalable success criteria or rubrics (like unit tests for code) that AI can then use to incentivize and improve model capabilities.

18. Expert-Created Evaluation Rubrics

Engage domain experts (e.g., lawyers for legal tasks) to create detailed rubrics that define success criteria and allow for effective scoring of AI model outputs, guiding model improvement.

19. Hire High-Skilled AI Evaluators

When training AI models, prioritize sourcing and assessing highly skilled professionals (e.g., experienced software engineers, lawyers, doctors) who can effectively evaluate and interpret complex model capabilities.

20. Evals as Sales Collateral

Use evals not only as internal guides for researchers to build model capabilities but also as external sales collateral to demonstrate the efficacy and practical value of your AI products.

21. Measure AI Automation Efficacy

For enterprises, build systematic tests or rubrics to measure how effectively AI automates your company’s core value chain, as this is a prerequisite for effective AI application.

22. Treat Evals as PRDs

Treat AI model evaluations (evals) as essential product requirement documents (PRDs) to guide development and measure success, similar to how researchers use them to make small improvements.

23. PMs: Leverage AI for Productivity

If you are a product manager, learn to leverage AI to significantly increase your productivity and output, as this will position you extremely well in the evolving job market.

24. Use AI for Writing & Thought Partnering

Utilize AI tools for writing documents and engage with them as a thought partner to reason through problems and gain advice, enhancing your thinking process.