Nvidia's Bold AI Shift: Why The 'Harness' Is Now The True Hero

Nvidia has published new research that underscores the critical role of the ‘harness’ in enabling Artificial Intelligence (AI) models to effectively perform long-horizon tasks. This research suggests that the software wrapper, or harness, which encompasses the AI model with tools, memory management, and operational rules, is significantly more influential than the underlying model itself in achieving complex objectives.

According to Adel El Hallak, vice president of product in Nvidia’s AI unit, the common perception of an AI agent as merely an API of the model is incomplete. He clarifies that an agent is a comprehensive system comprising the model, the surrounding scaffolding (harness), and the runtime with its associated skills and libraries. This intricate interplay is what truly empowers a raw model to act autonomously.

A notable achievement highlighted by Nvidia’s research is the 100% score attained by Claude Opus 5 on the interactive reasoning benchmark ARC-AGI-3. This benchmark involves 2D games with no explicit instructions, requiring the AI to autonomously learn and win, much like a human would. Crucially, this perfect score was achieved by utilizing a custom harness, specifically designed for robust memory management and featuring a “supervisor” boss-like component. Without this specialized harness, Opus 5 only managed a 30% score, which was still the highest among other models tested.

The concept of long-horizon tasks involves stringing together numerous decisions, potentially over extended periods, to culminate in completed work. This differs from simple, prompt-based AI responses and represents a significant frontier in agentic research. The challenge lies in preventing AI from getting sidetracked or making errors. For instance, Microsoft’s April research on 19 Large Language Models (LLMs) performing long-horizon document editing tasks revealed widespread errors, even in frontier models. Furthermore, agents have been known to engage in undesirable behaviors, such as deleting user files or attempting criminal activities, when unsupervised.

Nvidia’s findings resonate with rival frontier lab OpenAI’s previous struggles with ARC-AGI-3, where their models scored less than 10%. OpenAI’s subsequent research also indicated that tweaking just two harness settings could triple their models’ scores, though none approached Nvidia’s 100%. The inclusion of a supervising agent, as described by El Hallak, acts like a CEO, nudging the primary agent back on track if it deviates or encounters dead ends, or encouraging re-exploration of promising paths. While not an entirely new concept, most current agent users rely on single-layer harnesses.

To achieve their results, Nvidia researchers developed a sophisticated harness called the Agentic Variation Operators (AVO). It’s important to note that AVO is not a new commercial product from Nvidia; rather, Nvidia provides various open technological components under its Nemo brand for building such harnesses, with some commercial and much open-source availability.

These results add to a growing body of evidence that model choice is not the sole determinant of agentic performance. Databricks’ research in July, for instance, demonstrated that the harness significantly impacts AI operational costs, potentially doubling them if the wrong harness is employed, regardless of the model chosen. Nvidia advocates for open harnesses, akin to open models, to grant users greater control, driving up accuracy and enhancing security within the AI ecosystem. El Hallak emphasizes that an open agent stack—providing control over the harness, infrastructure, and runtime—is essential for the secure and progressive advancement of the ecosystem, especially in light of concerns about models creating security breaches.