Which examples belong in the prompt? A preprint tests Jev-style models

Anyone who classifies text with a language model often adds a few already solved examples to the prompt. This approach is called in-context learning, and the result depends heavily on which examples the model receives. A preprint by three authors from Jilin University in China, published on 31 October 2026, examines whether a decision model that estimates how useful each candidate is can help choose them. S81 It is a preprint, meaning work that has not yet gone through peer review. For developers the question is practical: better chosen examples can raise the accuracy of a small, cheap model without changing the model itself.

This is not the hosted Jev from TypeSafe

The authors refer to Jev from TypeSafe AI as a decision model that returns probabilistic judgments. However, the paper does not state that it worked with its API. In the experiments it used two open third-party models published on Hugging Face: NeoJev with 4 billion parameters and JEV with 9 billion. S81 The results therefore do not speak directly about the jev-1.14.0 model that TypeSafe offers through its API. S01 We also covered how teams build their own decision models in the article on the PayPal.AI team's preprint.

How the selection worked

The method has two steps. First, a standard sentence embedding model finds the 20 training examples closest to the input. Then the decision model answers a yes/no question for each of them: would it be a useful and relevant demonstration? The probability of a "yes" answer serves as the usefulness score. S81 The authors do not simply take the final set of examples in ranked order; they search for the best combination with beam search of width 4. S81

The catch is cost. Every candidate means another call to the decision model, which the authors describe as a substantial computational burden. They therefore proposed distillation: the ratings from the JEV-9B model served as training data for a small MLP neural network that then estimates usefulness on its own. They named this network DisJEV, and the equivalent network learned from the NeoJev model DisNeoJEV. S81

For comparison, the authors used selection by semantic similarity (kNN) and two other methods, ConE and DiSP. The tasks covered question classification (TREC), sentiment (SST-2, SST-5), news topics (AG News) and the relation between two sentences (MNLI). The examples were given to Llama3.2-1B, Qwen3-1.7B, Llama3.1-8B and Qwen3-8B, always four in the main experiment. S81

What came out

On all four models DisJEV had a higher average than selection by similarity, by 2.99, 3.68, 0.43 and 1.68 points. On Qwen3-1.7B it was 77.12 versus 73.44. S81 The DisNeoJEV variant did slightly better still in this test, reaching 77.68 on the same model. S81 According to the authors, ConE and DiSP brought inconsistent improvements and on some models even made results worse. S81

The average, however, hides large differences between tasks. On Qwen3-1.7B, DisJEV brought a marked jump on TREC (85.20 versus 72.40) and on MNLI (74.85 versus 69.95). On SST-2 and AG News, by contrast, it ended just below selection by similarity. S81 On the larger models DeepSeek-V4-Flash and Qwen3.8-Flash, which the authors called through APIs and evaluated on 500 samples, the gain was under two points. The authors themselves call it relatively modest. S81

The small network agreed with the original decision model on the order of candidate pairs in approximately 70% of cases. S81 When the authors trained it on only four tasks and evaluated it on the fifth, it trailed the full version by 1.43 and 1.70 points but still stayed above selection by similarity. S81 The authors also write that effectiveness depends on the specific language model and on the nature of the task. S81

What this means for a Slovak online shop

If a Slovak online shop sorts customer messages with a small language model and a few examples in the prompt, the paper suggests that the most similar examples are not necessarily the most useful. At the same time it shows that the benefit varies from task to task and is small with stronger models. S81 If you build the sorting directly on a decision model, look instead at the intent routing pattern.

What follows is JEV.sk's own suggestion, not a result of the paper. With the hosted Jev, a similar rating could be asked as a yes/no question for each candidate. TypeSafe charges USD 0.042 per million input tokens, and output is free. S01 Each candidate, however, is a separate rating with its own input, so costs grow with the number of candidates per message. You can make an estimate in the cost calculator. Before deploying such a layer, compare on your own messages whether it beats plain selection by similarity at all.