Jev in AI agents: faster decisions do not automatically mean better results

Anyone planning to replace the next-step selection in an AI agent with a low-cost decision model should look at more than the price of a single decision. That is the lesson of a new preprint by authors from Nanjing University and the Technical University of Munich, submitted to arXiv on 9 October 2026. They plugged the jev-1.13.0 model into five agent families and measured whether cheaper and faster decisions translate into successfully completed tasks. S47 It is a preprint, meaning work that has not completed peer review.

How the experiment was set up

The authors used two test environments. AndroidWorld contains tasks in mobile apps, and the study took 116 task types from it. ALFWorld simulates household chores, and the authors used all 134 tasks from one validation split. S47 In the first environment they tried the T3A, AutoDroid and FastAgent agents, in the second the ReAct and Reflexion agents. Each agent had an original version and a version with Jev. ALFWorld also had a control variant in which a language model was given the same list of options to choose from. S47 Each configuration was run three times, giving 4,500 runs in total. S47

Jev replaced only the action-selection step, with one request per step. The description of the environment, the history and the execution of the action itself stayed as before. Free text, summaries and reflections were still produced by the language model gpt-5.6-luna, which also made the decisions in the original agent versions. S47

The decision got cheaper, the task not always

For individual decisions the savings were large. Across all five agent families, the average cost of a decision fell by 66.1 to 78.4% and its time by 73.9 to 81.5%. A decision took 1.04 to 1.75 s instead of 4.13 to 8.13 s. At the same time, the number of tokens per decision rose by 53.5 to 134.9%. S47

The share of successfully completed tasks, however, fell for some agents. In ALFWorld the gap was small: ReAct reached 96.02% instead of 97.51% and Reflexion 98.01% instead of 99.75%. S47 On mobile tasks the drop was more noticeable. T3A fell from 57.47% to 44.25%, FastAgent from 50.00% to 31.90% and AutoDroid from 18.68% to 16.38%. S47 According to the authors, only the drops for T3A and FastAgent were statistically clear; for the other agents the interval of the difference includes zero. S47

Part of the savings was absorbed by auxiliary work of the language model. In the ReAct agent, decision time shrank from 68.31 s to 17.25 s, but 17.23 s of auxiliary reasoning was added. S47 The authors therefore also calculated the cost per successfully completed task, including failed attempts. ReAct became cheaper, from 7.93 to 3.96 thousandths of a dollar (mUSD), and AutoDroid from 27.93 to 9.93. For T3A, by contrast, the cost per success rose from 68.70 to 73.97, although the interval of this change also includes zero. S47 The time needed for one success grew for T3A from 370.6 s to 430.2 s and for FastAgent from 239.4 s to 377.8 s. S47

Where the agent with Jev failed

The authors manually went through the traces of selected runs. In tasks where the Jev version repeatedly lost, failed recovery from an error appeared more often: in 40 of 57 runs with Jev versus 3 of 57 for the original version. Ineffective search occurred in 29 runs versus 2. S47 Typical mistakes included picking the wrong object or an ambiguously labelled switch, sticking with an assumption the environment had already contradicted, and declaring a task done prematurely even though the goal had not been met. S47

The Reflexion agent is instructive. With Jev it succeeded on the first attempt in 95.27% of cases, the original version in 93.53%. Of the failed first attempts, however, the Jev version fixed 11 of 19, while the original fixed 25 of 26. S47 Faster selection thus helped in the normal course of work but coped worse with situations where something went wrong.

What the preprint does not measure

The authors examined only selected agent families in mobile apps and household tasks. The agents received a text description of the environment and chose from a limited set of actions. Three repetitions on the same tasks capture only the variation between runs, not behaviour in new environments. Prices and times also depend on model versions and service conditions. S47 These are not e-commerce tasks, and JEV.sk did not measure any of this.

What this means for a Slovak online shop

The authors recommend judging Jev-type models by successfully completed tasks, the ability to recover from errors and the cost per delivered result, not only by the speed and price of a decision. In their view, Jev suits clearly specified decision roles. They consider most promising those systems that make routine choices cheap and concentrate verification on decisions that threaten task completion. S47

JEV.sk's own suggestion for an online shop: Jev can decide which queue a customer message belongs to or what type of return claim it is, but whether an order has really been completed should be checked by code against the state in the system, not by the model. Declaring a task done prematurely was among the typical failures in the preprint. S47 If you build an agent that proceeds through several steps, measure the whole flow: how many cases it actually resolved and what one resolved case cost. How to use the model's confidence when deciding whether to continue or hand a case over is explained in the guide confidence and calibration. You can estimate the cost of the decisions themselves in the cost calculator.