Vercel adds a fallback model for uncertain Jev decisions: how it works

On 6 October 2026, Vercel added a feature to its AI Gateway service that directly concerns the Jev decision model: when the first model's answer fails a confidence condition you set yourself, the gateway reruns the whole decision with another model. The feature is still in beta. S43 A safety net that would otherwise have to live in your own code can now be described with a single object in the request. It does, however, come with rules you should know before putting it into production.

Two kinds of fallback models

AI Gateway can also switch to another model when the first one fails to execute. The new feature covers a different case: the first model answers successfully, but the answer is uncertain. S44 In the providerOptions.gateway.models list, the two cases differ by the shape of the entry. A plain model name is a fallback for errors. An object with a when field is a conditional fallback that runs only when the condition holds for the first model's answer. Requests without a conditional object behave as before. S44

A request may contain only one conditional object, and it must come first in the list. Plain error fallbacks may follow it. The gateway runs at most two successful stages: after the rerun it does not check the condition again and returns the fallback model's result without mixing in answers from the first stage. The fallback model receives the same state and all original questions, not just the uncertain one. S44

Conditions: confidence for Choice and Score, probability for Noul

Choice and Score questions use confidenceBelow. For yes/no questions (Vercel calls them Boolean; in the TypeSafe-compatible API they are Noul), you instead set an inclusive probabilityBetween range for the probability of a positive answer. confidenceBelow cannot be used with a Noul question; the gateway rejects such a request with a 400 error before the model runs. S44

If you omit the question field, the condition applies to all questions of that type, and one uncertain answer is enough to trigger the fallback. Conditions can be combined with any, all and atLeast; each group may hold 1 to 20 conditions, nested at most five levels deep. S44 An important detail: when a Choice or Score answer has no finite confidence value, confidenceBelow counts as met and the gateway records the reason confidence_unavailable. The condition is also met when the model refused to answer the question. S44

What a threshold of 0.6 really means

The examples in Vercel's documentation use a threshold of 0.6 for confidence and a range of 0.4 to 0.6 for probability. S44 The value 0.6 also appears in the confidence routing pattern in TypeSafe's documentation. Using a voice banking example, it sends cases below this level to a human and requires higher confidence for sensitive actions. S35

Keep in mind, though, that confidence is not the probability of the winning option. TypeSafe describes it as a measure of how concentrated the probability distribution is, on a scale from 0 to 1. For Choice it is computed as (p_max - 1/n) / (1 - 1/n), where n is the number of options. S06 JEV.sk's own calculation using this formula: with three options and a winner probability of 0.6, confidence comes out at 0.4, so with a threshold of 0.6 such an answer would go to the fallback model.

TypeSafe's documentation stresses that the right thresholds depend on the domain. S06 Vercel recommends recording the confidence values at which the first model makes mistakes and setting the narrowest threshold that catches the errors worth paying for a second stage. S44 We cover this in more detail in the confidence and calibration guide.

Cost, latency and the zero-confidence trap

When the fallback runs, you pay for both stages. Because they run one after the other, their latency adds up as well. If the fallback model fails, the request ends with an error, the first model's result is not returned, and the first stage is billed anyway. S44 According to TypeSafe, Jev itself costs 0.042 USD per million input tokens, and output tokens are not charged. S01 The fallback model, however, is billed at its own rates, so the final cost depends mainly on how often the condition is met. The pricing and limits guide helps with an estimate.

If the fallback is a language model, its answer has no confidence value, because it returns a single structured value without a probability distribution. In the TypeSafe-compatible API (POST /typesafe/v1/systemone), the gateway then returns confidence: 0 and empty probabilities. According to the documentation, this means the value is not available, not that confidence is zero. S44 Code that sends low-confidence results for manual review could, without changes, treat every such result as uncertain. You can find out which model answered from the model field or from provider_metadata.gateway.routing; when the fallback ran, also from the x-ai-gateway-decision-fallback-* headers. S44

What this means for a Slovak online shop

The examples in Vercel's documentation show customer requests being sorted into teams, such as billing, technical problems and account access. S44 In an online shop, this is a task similar to sorting messages into return claims, shipping and invoices: Jev handles the clear cases and another model takes over the unclear ones without any retry logic of your own.

Check three things before deployment. First, store the triggeredBy list from the response so you can see how often and for which questions the fallback runs; cost and latency depend on it. S44 Second, if you read results through the TypeSafe-compatible API, handle zero confidence after a fallback separately. Third, the official TypeSafe SDK passes the providerOptions field at runtime but does not yet include it in the static request type, so it has to be supplied through an intermediate object. S44 How to call Jev through APIs and gateways is described in the API integration guide. We discussed a similar cascade from a research perspective in the article Jev as a low-cost first judge.

The feature is in beta and JEV.sk has not tried it. Everything above is based on Vercel and TypeSafe documentation.