In the race to build superintelligent AI, high-caliber data is everything. Mercor, a data annotation startup, has emerged as one of the fastest-growing companies in history by offering something most rivals don’t: expert-labeled evaluations that benchmark the capabilities of large language models.

AI evaluations (“evals”) are structured assessments that measure how well a model performs on tasks within specific domains. Instead of low-wage workers tagging data, Mercor hires doctors, engineers, lawyers, investment bankers, and other professionals to judge model outputs for quality and accuracy. 

A software engineer, for instance, might evaluate code for security flaws or functional completeness, then design a rubric developers can use to benchmark improvement. Those rubrics become the scoring guides that machine learning engineers deploy to grade model responses, flag failures, and define what a “good” answer looks like. 

Put simply: evaluations form the baseline that AI companies use to steer their models towards outputs they desire.

“We are in the age of AI evaluations,” Mercor co-founder and CEO Brendan Foody said on a September episode of Lenny’s Podcast.

Since launching in January 2023, Mercor has inked deals with frontier model developers like OpenAI and Anthropic, as well as six of the “Magnificent Seven” tech giants, including Alphabet, Amazon, and Google. In just 17 months, Mercor saw its annual revenue run rate surge from $1 million to $500 million. The company claims it now employs more than 30,000 experts, paying contractors an average rate of $95 an hour to test and score model performance. As of October 2025, Mercor’s valuation sits at $10 billion, with investments from Felicis, Benchmark and General Catalyst.

Its meteoric rise is emblematic of a strategy built on the need for high-quality data, confident leadership, and a bet that expert-driven evaluations will define the next phase of the AI arms race. But as companies pour more money into AI training data and see their models evolve, a growing number of analysts and investors are wondering whether Mercor may be flying too close to the sun.

The rise of Mercor

In January 2023, three college dropouts — Foody, Adarsh Hiremath, and Surya Midha — launched Mercor as an AI talent marketplace connecting programmers with U.S. companies. Foody and his co-founders quickly recognized a shift underway in the AI ecosystem. After meetings with OpenAI and xAI, they pivoted the company toward expert-driven evaluations, betting that model developers would increasingly rely on professional judgment to advance their technologies. Human expertise, they believed, would unlock the next wave of AI performance.

As the company built out its evaluation platform, contracts with top AI labs followed. A turning point came when an undisclosed rival tapped Mercor to source more than 1,000 contractors for its own projects. The operation quickly unraveled under a wave of support tickets from unpaid workers. For Foody, it was a wake-up call about the inefficiencies and vulnerabilities of middlemen: a chance to bring more of the process in-house.

“The wealthiest companies in the world are willing to spend whatever it takes to advance model capabilities,” Foody told Lenny’s Podcast.

That willingness to spend has fueled Mercor’s aggressive expansion. Today, the company claims it pays out roughly $1.5 million a day to its evaluators. And with growing demand from AI labs seeking better performance benchmarks, the expectation is that spending will only increase.

But with high pay comes high standards. One annotator, who worked on Mercor’s nutrition and dietetics projects, said the company enforces strict expectations. “[Mercor] has very high standards, and they will offboard people really quickly if they’re not submitting quality work,” the contractor, who asked to remain anonymous, told Big Think.

It’s clear that the need for Mercor’s highly-skilled talent has never been higher. In 2025, that momentum reached a new peak. In February, Mercor closed $100 million in a Series B, bringing its valuation to $2 billion. Then, in mid-June, Meta invested $14.3 billion in Scale AI, triggering a wave of customers to turn to rivals like Mercor. More than six months later, Mercor raised an additional $350 million in Series C led by Felicis, multiplying its valuation five-fold to its current valuation of $10 billion. 

“Brendan has shown remarkable focus and discipline,” Sundeep Peechu, who led Mercor’s last two rounds of investments as Felicis’s managing partner, told Big Think. “He’s a rare mix of visionary and operator.”

Mercor’s AI evaluations, Peechu adds, have become “cornerstone infrastructure” in the AI ecosystem as the economy leans into what Mercor calls a “reinforcement learning (RL) environment machine,” where human feedback continuously shapes the trajectory of AI systems.

But not everyone is convinced the company can keep up its pace. 

The long-term growth question

Tech analysts and investors agree that Mercor’s business will continue to grow in the short-term. But some experts say its long-term growth hinges on how quickly AI capabilities develop. 

John Bersin, founder of The Josh Bersin Company, a market research and advisory firm, questions whether Mercor’s experts are truly top-tier or simply skilled contractors helping models edge forward. He doubts that seasoned professionals have the time or incentive to spend hours evaluating AI outputs. 

“A true expert is likely to be either too busy or too financially successful to spend days annotating data,” Bersin told Big Think. “It’s more than likely that the labeling experts are simply contractors helping the system become a little smarter than before.”

Scott Chou, co-founder and managing partner of the ESO Fund, says the company may potentially run out of qualified talent to hire. “I don’t really get Mercor,” Chou told Big Think. “It seems a lot like Scale, which smartly exited before the bottom came out. There’s value and sales to be had in the near term. But they’re operating in a space where sustaining growth over time will be difficult.”

Edward Hartman, partner at Simon-Kucher, a growth consulting firm, sees risk either way. He says that if AI gets good fast, Mercor could be “undercut by an oversupply of newly displaced professionals.” If it doesn’t, the need for human oversight could disappear altogether. Either way, he said, the infrastructure might outpace the revolution it’s built to support.

In response to these concerns, a Mercor spokesperson told Big Think that there are no signs the talent pool is shrinking and that AI is creating new jobs and skills in the workplace. In other words, human judgment still has a critical role to play in shaping machine intelligence, and there’s a long way to go before AI will reach peak capabilities.

“I know there’s been some executives at big labs that say we’ll have superintelligence in three years,” Foody said on Lenny’s Podcast. “But I think the truth is that it’s a longer road.”