Jev tests: what the numbers prove and what they don’t

Three different kinds of evidence

A marketing promise, an external experiment and your own operational measurement must not look the same. Every test on JEV.sk should state the author, date, model, sample, language, setup and limitations. A test proposal must not be presented as a finished benchmark.

Banking77: classification in English

On 22 September 2026, OpenRouter compared Jev 1.13 and Claude Opus 5 on 3,080 Banking77 utterances (40 per intent). Jev achieved an accuracy of 81.0% (interval 79.6–82.3%), Opus 84.4% (83.1–85.6%). The median latency was 175 ms for Jev and 2,266 ms for Opus; the 95th percentile was 270 ms and 3,004 ms. The cost per thousand requests came to $0.11 and $2.42. S18 S29

According to the article, a cascade of both models with a threshold of 0.90 achieved 84.0% at $0.69 per thousand requests. This is the result of a single English classification task, published by a provider that offers access to both models; it is not a universal accuracy figure. S29

A Czech test on vibecoding.cz

On 19 September 2026, Patrick Zandl described using Jev for internal linking across 209 articles on vibecoding.cz, 139 of which had no link from any other article. After a pre-selection of the 15 nearest candidates by embeddings, Jev assessed 3,135 pairs. According to the author, the entire first run cost $0.16, the median latency was 309 ms and the 95th percentile 414 ms. On a sample of 150 pairs, Jev agreed with the Opus model in 126 cases (84%), and according to an evaluation by the Sonnet model, 99% of the links recommended above the chosen threshold were genuinely related. S27

This is one website, one task and Czech, not Slovak. The reference was other language models, not manual evaluation by humans. The test therefore shows practical usability and costs, not general accuracy. S27

Vendor data

TypeSafe’s launch materials present very substantial speedups and cost reductions in selected workflows. On JEV.sk, we report them as vendor claims, not as a promise to every customer. S10

Our test, not yet carried out

We propose a test on Slovak products and messages: the same manually established ground truth, a stored version of the task definition and the same inputs for all models. The results should also show failures and the escalation rate. Instead of the price per token, we prefer the cost per correctly handled case.

JEV.sk has not yet carried out its own tests via the API. That is why you will not find a “tested” badge, star ratings or reviews here.