Structured memory as context for Jev: a preprint finds a small, unproven gain

Three authors with email addresses on the abbycare.ai domain published a preprint on arXiv on 3 November 2026 asking whether Jev benefits when an ordered block of facts is added to the free-text description of a case. S82 They tried this on synthetic patient cases, but the question is a general one: should you send only raw text into the state, or also data that has been processed in advance? This is a preprint, the text does not mention peer review, and the authors themselves describe the result as directionally positive, not as clinical validation. S82

How the test was set up

The authors prepared 30 fully synthetic cases from twelve medical domains, with 120 questions: 56 multiple-choice, 36 yes/no and 28 ratings on a five-level scale. S82 No real patient records were used. S82 Each case ran in two variants. In the first, Jev received only the text description of the case. In the second, a memory block was added: a list of conditions or medications. Each entry has a normalised label, a status (active, history of, resolved), an onset, a certainty from 0 to 1 assigned by the authors, a source form, a negation flag and an optional note. S82 All questions for one case went in a single request, giving 60 paired calls in total. S82 The guide Choice, Score and Noul explains how the question types differ.

One important detail: the memory was not extracted automatically from the text. The authors generated it synthetically and describe it themselves as an upper bound, because real extraction would likely be noisier. S82 They called the Jev early-access interface and did not record the exact model revision in the results file. S82 So it is not known whether this was version jev-1.14.0, which the jev-latest alias points to today. S01

Results: better probabilities, unproven accuracy

Accuracy rose from 108 of 120 answers (90.0%) to 112 of 120 (93.3%). S82 The confidence intervals overlap: 84.6% to 95.4% against 88.9% to 97.8%. With memory, five questions changed from a wrong answer to a correct one and one went the other way. The exact McNemar test gave p = 0.219, and the authors acknowledge that the sample is small for such a comparison. S82

The probabilities changed more clearly. Log loss on multiple-choice questions fell from 0.258 to 0.175, the Brier score on yes/no questions from 0.054 to 0.022, and the mean deviation on the scale from 0.521 to 0.469 levels. S82 On the scale, the share of answers within one level rose from 85.7% to 92.9%. The authors, however, treat the breakdown by question type only as a prompt for further research. S82

In the 0.9 to 1.0 confidence band, the variant without memory had 55 answers and all of them were correct. The variant with memory had 65 answers in the same band, with 98.5% correct. S82 Memory therefore moved more answers into the highest confidence band. The other bands were sparse, though, and the authors state explicitly that they did not establish formal calibration. S82

The price was paid in tokens. Mean input per call grew from 553.9 to 863.9 tokens, an increase of 310.0 tokens or 56.0%. Mean latency stayed practically the same: 0.51 s without memory and 0.50 s with it. S82

Limitations the authors list

The list of limitations is long and should be taken seriously. The data are synthetic and may not capture the ambiguity, errors or missing information of real records. The correct answers were prepared by the authors without review by a clinician. The test used one model and ran once. S82 The authors also point out that the schema itself is part of the intervention, so the gain cannot be attributed to structure alone as opposed to a human having normalised the data in advance with expert knowledge. S82 As a next step, they propose replacing the reference memory with the output of real extraction and comparing it with raw forms, retrieved passages and summaries. S82

What a Slovak online shop can take from this

A Slovak online shop does not deal with medical records, but it makes a similar choice with every question it sends to Jev: send only the text of the customer's message, or add ordered data from the order system as well. JEV.sk's suggestion based on this preprint: if you add data, give it, as in the test, a clear status and an explicit negation, for example "complaint: closed" or "return: not requested", rather than a loosely copied history. The preprint, however, tested error-free memory, so the benefit for data extracted automatically from emails needs to be checked on your own cases. S82

The costs are easy to quantify. TypeSafe charges 0.042 USD per million input tokens, and output is free. S01 With 310 extra tokens per call, as in this test S82, JEV.sk's calculation puts 100,000 calls at roughly 1.30 USD more. Plug your own numbers into the cost calculator. If you plan to use confidence to decide whether the system handles a case on its own, see the guide confidence and calibration. Higher confidence in this test is not evidence of good calibration on your data, and the preprint did not examine Slovak texts.