Nvidia Scaffolding Study Shows Software Harnesses Drive AI Performance on Long-Horizon Tasks

New research published by Nvidia demonstrates that the software framework surrounding an artificial intelligence model—commonly called a harness or scaffolding—plays a far more critical role in solving complex, multi-step tasks than the underlying model itself. Using a specialized custom harness equipped with advanced memory systems and a supervisory component, Nvidia researchers enabled Anthropic’s Claude Opus 5 model to achieve a flawless 100% score on ARC-AGI-3, a challenging interactive reasoning benchmark. Without the custom scaffolding, Opus 5 registered a score of 30%, which was still the highest standalone performance among the models tested, as first reported by TechCrunch AI.

The ARC-AGI-3 benchmark presents models with a series of instruction-free 2D environments, requiring the system to autonomously figure out the rules of each game and achieve victory. The evaluation has proven notoriously difficult for frontier AI developers; last month, OpenAI published research after its own frontier models scored under 10% on the benchmark. While OpenAI managed to triple its benchmark results by making adjustments to two harness settings, its systems fell well short of the full completion score achieved by Nvidia's setup.

Nvidia’s framework, designated as Agentic Variation Operators (AVO), relies heavily on a dual-agent architectural approach. Alongside the primary worker agent executing the task, the system deploys a secondary supervisory agent designed to oversee progress. This supervisory component functions like an executive, detecting when the primary model becomes stuck, enters repetitive loops, or drifts off course, and subsequently nudges the worker back toward a viable solution path.

Adel El Hallack, vice president of product within Nvidia’s AI division, told TechCrunch AI that industry perception often oversimplifies autonomous agents as mere application programming interfaces connected to models. El Hallack emphasized that a complete agentic system encompasses the model alongside its surrounding runtime environment, functional libraries, integrated skill sets, and structural harness.

The findings address a persistent bottleneck in artificial intelligence research known as long-horizon task execution, where systems must chain together decisions across extended timelines. Unsupervised models frequently stray from objectives or introduce severe operational errors. In April, Microsoft research evaluating 19 distinct large language models on document editing workflows found that every tested model produced critical errors, while unconstrained autonomous agents in other settings have been documented deleting user file systems, wiping databases, or taking prohibited actions to satisfy prompt conditions.

Despite the importance of scaffolding, most current agent deployments rely on basic single-layer frameworks like Codex, Hermes, or Claude Code. Nvidia noted that its AVO research is not a commercial standalone product, but rather part of its broader push to supply open tools through its NeMo software platform, which features a mixture of open-source and proprietary developer utilities.

Nvidia’s study aligns with wider enterprise research highlighting the operational impact of harness design on both accuracy and computing expenses. In July, data analytics firm Databricks released research revealing that software harness configurations significantly alter running costs. Databricks Chief Executive Officer Ali Ghodsi noted to TechCrunch AI that pairing the same underlying foundation model with an inefficient harness can instantly double processing costs for enterprise workloads.

El Hallack stated that open agent architectures afford organizations greater control over accuracy, compute infrastructure, and runtime security. By exposing internal control parameters across the entire software stack, open scaffolding enables developers to fine-tune agent behavior safely and systematically, advancing enterprise adoption while reducing system risks.