Public benchmarks can show how AI performs on tests. Enterprise buyers need to know whether those systems can be trusted in real work.

An AI model can top a public benchmark and still fail the professional question a company needs it to answer.

Companies buying AI are running into a central problem. Public benchmarks can show whether a system performs well on a standardized task. They are less useful for deciding whether the same system can be trusted inside a company’s own work, using its data, rules and oversight requirements.

Pearl Enterprise, which builds AI systems for professional services, has put numbers around that concern. In a new evaluation of leading AI models, the company compared model responses with expert-authored answers across business, health, law, pets and technology. The top model, OpenAI’s GPT-5.5, reached 72.7 percent expert alignment overall, according to Pearl’s leaderboard.

Andy Kurtzig, CEO of Pearl, said the result should change how executives hear claims about top-ranked AI systems.

“A C- student who knows they’re a C- student is manageable,” Kurtzig told Newsweek. “A C- student who’s convinced they’re an A+ student, and answers every question with total confidence, is the one who gets a company in trouble.”

Pearl’s findings need some context. The company sells AI systems for professional services, a category where expert review is central to the product. The leaderboard measures expert alignment, rather than a simple right-or-wrong score. According to Pearl’s public methodology, the AI models were given the same questions used to create expert reference answers, without tuning or post-processing. Their responses were then scored against those expert-authored answers on a 1-to-5 rubric using Anthropic’s Claude Sonnet model.

Pearl is not the only company raising this sort of concern. Executives working across AI security, software, compliance and enterprise deployment told Newsweek that higher public scores do not prove a system is truly ready for work where mistakes can be costly.

Rob Clark, president of Seekr, an AI company focused on model evaluation, said rankings can help companies compare systems, but they cannot substitute for testing a tool inside a specific business setting.

“A leaderboard ranking is not a trust decision,” Clark told Newsweek. “Confidence scores measure how sure the model is, not how right it is.”

Pearl’s domain results make the point more clearly when viewed within the same model. GPT-5.5 led the business category with 80.9 percent expert alignment, but Pearl’s data showed the same model falling to 68.8 percent in health and 62.1 percent in pets. The spread suggests companies need to ask a narrower question: whether a high-ranking model has been tested in the specific field where it will be used.

Companies rarely deploy AI for “general intelligence.” They use it for claims review, contract analysis, customer support, coding, security triage, sales operations or internal search. A broad score can obscure the weakness that matters most.

“A model is not ‘good’ or ‘bad,’” Kurtzig said. “It is good at some things and dangerous at others, and an aggregate score blends all of that into one number that hides exactly where it fails.”

Public benchmark scores often come from tests such as GPQA, which uses difficult science questions, or SWE-bench, which evaluates software engineering tasks. Zhenwen Dai, CTO and co-founder of Trent AI, an AI security company, said companies should be careful about relying too heavily on scores from public tests. They can be useful signals, he said, but they can also encourage companies to overestimate how well a model will perform in their own environment.

“A model may become better at the benchmark without becoming equally better at the real-world tasks a company actually needs to solve,” Dai told Newsweek.

Vendor demos and public tests can miss the details that matter once a system enters a company’s environment.

“Companies should evaluate AI at the level of the specific task or workflow, rather than relying on general model capability scores,” Dai said.

What a buyer deploys may be far removed from the base model that performed well in a public test. The same underlying technology can behave differently once it is embedded inside a product, connected to tools or given permissions inside a company.

Varun Badhwar, founder and CEO of Endor Labs, a software security company, sees the problem clearly in AI coding agents. His company runs the Agent Security League, a public benchmark for AI coding agents that extends research from Carnegie Mellon University. The benchmark tracks the gap between code that passes functional tests and code that is actually secure.

“The benchmarks aren’t wrong—they’re measuring the wrong axis for an enterprise buyer,” Badhwar told Newsweek. “They measure capability: Can the model do the task? They’re largely silent on the properties that actually create enterprise liability—is the output safe, is it accountable, is it correct in a way you can defend later?”

Badhwar cautioned against assuming a base model’s score carries cleanly into production. A company is using a full system, not an abstract model on a leaderboard.

“You are not deploying the model,” he said. “You’re deploying the model, inside a harness, pointed at your data, under your constraints.”

Human review also becomes more specific once AI is connected to tools, data and business processes. The point is not to have a person glance at every AI output. The question is where failure would be expensive, quiet or difficult to reverse.

Patrick Sullivan, vice president of strategy and innovation at A-LIGN, a cybersecurity compliance company, said evaluations are most useful when companies can turn them into evidence for how an AI system performs in real work.

“Benchmarks measure capability under clean conditions,” Sullivan told Newsweek. “Professional workflows are not clean.”

Professional settings add unclear inputs, changing rules, internal systems and records that may need to be inspected later by customers, auditors or regulators. Once companies put AI systems into those settings, Sullivan said, they also need assurance: proof of how the system behaved, what controls were in place and whether a reviewer can reconstruct the decision later.

“The right question isn’t ‘How well does this model perform?’” Sullivan said. “It’s ‘Can I produce evidence that it performed correctly, on my work, for the use case I deployed it against?’”

Several executives described the same basic standard: Review should rise with risk. A tool that drafts an internal memo does not need the same oversight as one that affects a payment, security decision, health recommendation, legal right or compliance record.

Badhwar said the review threshold should be tied to what the error could touch.

“Anything touching money, security, regulated decisions, or anything that’s hard to reverse gets a human, regardless of how confident the model is or how high the aggregate benchmark sits,” he said.

Expert review is also needed earlier, Dai said, when companies are deciding what to measure. A poorly designed evaluation can make a system look safer than it is.

“The quality of the evaluation framework directly affects whether the deployment decision is valid,” Dai said.

Clark pushed the concern into procurement. As capable, inexpensive models flood the market, he said, many buyers cannot answer basic questions about who built them, what they were trained on, how their behavior was shaped or where company data flows.

“You wouldn’t onboard a supplier into your physical supply chain without knowing its origin,” Clark said. “The same diligence has to apply to your AI supply chain—especially in regulated and federal environments.”

The concern has already moved beyond internal AI committees. This month, the U.S. government ordered Anthropic to block foreign nationals from accessing its Fable 5 and Mythos 5 models, citing national security concerns; Anthropic said it disagreed with the decision but disabled access to comply.

Kurtzig said companies need to draw clearer boundaries around where AI can act on its own and where the consequences of a wrong answer are too high.

“’Mostly right’ is the wrong bar,” he said. “A single bad answer can outweigh a thousand good ones, so the real question is not how often the model is right. It is what happens when it is wrong.”

Enterprise AI adoption is likely to keep moving quickly. The companies in the strongest position may not be the ones that simply pick the highest-scoring system. They will be the ones that can say where a tool works, where it needs review and where it should not act on its own.

Clark framed testing as the thing that lets deployment move faster, rather than the thing that slows it down.

“Speed is the payoff for testing the model against the job it’ll actually do, in the environment it’ll actually run, before you ship it—not after,” he said.