Skip to content
All posts

Measuring the Capabilities of AI Agents

Most AI benchmarks evaluate what large language models know. METR, an AI research organization, takes a different approach by measuring what AI models and agents can accomplish autonomously and how long they can reliably perform complex tasks.


METR measures this through the task-completion time horizon, which estimates the duration of tasks an experienced professional could complete that a frontier AI agent can successfully perform with 50% reliability. For example, a two-hour time horizon indicates the model can complete tasks requiring approximately two hours of expert human work about half of the time (https://metr.org/).

 MERT

METR's research suggests that this capability has doubled approximately every seven months over the past several years. If this trajectory continues, AI agents could eventually complete projects requiring weeks or even months of human effort. At that stage, AI would extend beyond information retrieval to executing increasingly complex, multi-step workflows with limited human supervision.

These results should be interpreted carefully. METR evaluates well-defined tasks under controlled conditions, while real-world business environments often introduce greater uncertainty and complexity. As a result, the measured time horizon should be viewed as an indicator of current capabilities rather than a definitive limit.

The broader trend remains significant. As AI agent capabilities continue advancing, technology services firms will have increasing opportunities to redesign delivery models, automate more complex workflows, improve operational efficiency, and develop new AI-enabled service offerings.

Elaxtra Advisors is an M&A and value-creation advisory firm that assists institutional investors, private equity-owned platforms, and strategic acquirers invest and create value in worldwide technology services companies. Please contact us to explore potential partnerships.