OpenAI's Secret Weapon: The 'Jev Clone' to Tame AI Swarms

A significant revelation emerged from OpenAI’s Dev Day event, where CEO Sam Altman introduced the company’s new “Decisions API.” This API appears to offer capabilities akin to Jev, a model launched earlier by TypeSafe AI, specifically engineered for software automation. Jev functions as a highly efficient classifier, built upon a large language model (LLM), enabling developers to provide it with a selection of choices that it then outputs as probabilities with remarkable speed and cost-effectiveness. OpenAI’s Decisions API seems to be a parallel product, designed to equip the lab’s Luna model with a predefined array of options for selection, such as classifying images into categories or determining various agent behaviors. Altman underscored that by concentrating the model on such specific choices, it can achieve extreme speed while preserving essential functionalities like image understanding, broad language support, and critical safety protections.

The announcement prompted a playful response from TypeSafe AI’s CEO, Diogo Almeida, a former OpenAI engineer and co-inventor of reinforcement learning, who quipped on X about the commencement of “clone wars.” Almeida further suggested that OpenAI’s interest in this domain could signify that “building in a System One compatible way is the future.” TypeSafe AI defines “System One” as fast, intuitive thinking, contrasting it with “System Two,” which denotes deliberate reasoning. The underlying implication is that conventional LLMs, in their current form, are not always the optimal solution for many software applications due to their comparative slowness and expense. Developers have already been leveraging Jev to augment existing LLMs, discovering substantial improvements in speed and cost efficiency as a result.

While the full extent of the Decisions API's similarity to Jev remains somewhat unclear, given its limited preview release and the absence of extensive developer testing observed by TechCrunch, market interest is palpable, evident from discussions across social media platforms like X. It is also important to note that OpenAI’s offering is not unique; other startups are actively developing and rolling out comparable decision-making models, indicating a burgeoning trend that major tech entities will likely continue to embrace. A crucial question surrounding these decision models is the accuracy of their outputs and how well they are calibrated to real-world scenarios. Almeida asserts that TypeSafe AI's competitive advantage, or ‘moat,’ lies in the synthetic data it generates to produce statistically robust and useful outputs. He emphasized to TechCrunch that while achieving speed and low cost is straightforward, true intelligence is the challenging component, and his company's guiding principle is to continually advance the intelligence-per-dollar Pareto curve.

Within a mere few weeks, the future utility of these models has become evident, with one particularly promising application being the monitoring and security of AI agents. Following a series of incidents involving misbehaving AI agents on the open internet, OpenAI has implemented new security protocols that involve using a separate model to detect undesirable actions, albeit at a “significant compute cost.” Shapor Naghibzadeh, a seasoned cybersecurity professional and head of the startup QueryStory, believes that a model like Jev could facilitate such monitoring far more economically. Naghibzadeh demonstrated this concept at a recent hackathon, where he built a demo utilizing Jev to scrutinize each agentic action against its assigned task. This system effectively blocked actions deemed highly confident to be malicious, flagged others for review, and permitted the remainder. Theoretically, this type of monitoring could have prevented incidents such as the Hugging Face event. The cost disparity is striking: such monitoring could cost $2.94 with Jev, versus $372 when employing a frontier LLM. A key insight is that Jev's affordability makes it viable to run on every agentic action, thereby introducing a vital layer of review that could significantly enhance the overall reliability of AI agents. This outcome aligns precisely with TypeSafe AI's initial aspirations, a value proposition now clearly recognized by OpenAI as well.