Jev for product recommendations: a preprint favours a small trained model

Italian and German researchers have published Decision4Rec, a preprint that builds a recommender system around a decision model. Among the models tested was Jev in version jev-1.13.0, which they called through its API. S67 For online shops the result is interesting mainly because of what did not work: the decision model used as released added little to a standard recommender, and a small open model trained on real user choices outperformed Jev in most settings. S67 Version v1 has been on arXiv since 19 October 2026, and it is a preprint that has not completed peer review. S67

How the system is put together

The authors, from Università degli studi di Bari Aldo Moro and Otto-von-Guericke University Magdeburg, define a decision model as a language model that answers a typed question with a probability for each option and writes no text. S67 The system has four parts. A knowledge graph rewrites behaviour logs as short statements on the shop's own scale, for example that an item sells in the top 10%; this graph is not trained. A group of agents proposes candidates, and an auxiliary ranking model keeps 30 of them. The decision model then chooses among the candidates based on their text, the statements and information about which agent found them. Finally, its answers are combined with the ranking model's score. S67

Each user requires one request with two questions of 30 options each, and the statements for one option take about 20 tokens. S67 Besides Jev, the authors tried the open Kev models and their own versions trained with LoRA, ranging from 0.8B to 27B parameters. S67 They trained on three source domains (Amazon Video Games, Amazon Musical Instruments and a sample from Steam), while the Amazon Industrial and Scientific and MovieLens-1M domains were kept for validation. The metric was NDCG@10 on 2,000 test users in each domain. S67

Where the gain came from

With a trained 4B model, the full system reached an NDCG@10 25% higher than the best trained baseline on Amazon Video Games and 36% higher on Musical Instruments. On the dense Steam dataset the difference was +1% and not statistically significant. S67 Most of the gain, however, already came from the agent group with the ranking model: without the decision model, the figures were +17%, +26% and −1%. S67 On the Industrial and Scientific domain, which the decision model did not see in training, the system with the 4B model reached 131% of the SASRecF baseline's result. S67

The released Kev-4B model without training, when reranking the top 50 suggestions from the SASRec model, was not significantly better on any dataset and helped roughly the same users as a random order. S67 When it was given customer behaviour in the form of short statements, however, it improved by 19 and 45% in two of three domains. S67

Jev against a small trained model

The authors' own trained Kev-0.8B was better than the released Jev in eleven of fifteen settings, by 14 to 43%, and worse in none. The trained Kev-4B outperformed Jev in fourteen of fifteen. S67 The released Jev and the released Kev-4B did not differ significantly in ten of fifteen settings. S67 The authors conclude that training on real choices mattered far more than the size or price of the model. S67

Jev answered in 310 to 340 ms, and a thousand users cost 0.15 to 0.22 USD at the price of 0.042 USD per million input tokens that applied during the experiments. The whole Jev section cost 4.89 USD. S67 The trained Kev-0.8B needed 157 to 167 ms per user on a single GPU with 24 GB of memory. S67 The authors also resent the same 1,200 requests a few hours later. Only 0.3 to 0.7% of the answers matched, and the first choice changed in 7% of requests. The effect on accuracy was small, but the result of the hosted model does not depend on the request alone. S67 They see further limitations in that users cannot fine-tune Jev, its outputs cannot be used to train a cheaper model, customer data leaves with every request, and the vendor can retire a version. S67

What this means for a Slovak online shop

The largest gains appeared in sparse catalogues, with an average of 16 to 32 interactions per item. On the dense Steam data and the plain version of MovieLens-1M there was no gain. S67 A smaller online shop with a wide range and few purchases per product resembles the sparse datasets more closely, but the preprint did not test this directly.

JEV.sk's own suggestion: if you are considering Jev for recommending related products, do not expect an improvement from simply reranking the results of an existing recommender. According to the preprint, short statements about sales performance and customer behaviour, and suggestions from several sources, helped more. S67 If a customer should see the same recommendations on a return visit, store the result, because the same request can turn out differently. S67 You can work out the cost of calls with the pricing and limits guide, and a broader overview of use cases is on the Jev for online shops page.

Limits of the preprint

The target item was among the 30 candidates for only about 9 to 25% of users, so reranking cannot help the rest. S67 Each model was trained once, the authors implemented the baselines themselves, and validation on new users is still in progress. S67 The version tested was jev-1.13.0. The jev-latest alias now points to jev-1.14.0, available since 14 October 2026, at the same price of 0.042 USD per million input tokens. S01 Whether the newer version would perform differently is not examined in the preprint, and JEV.sk has measured none of this.