Many questions in one Jev request: the fan-out pattern in practice

The TypeSafe documentation includes a pattern called speculative fan-out. Instead of a chain of calls in which the second question waits for the result of the first, Jev receives all questions in a single request, including those whose answer may end up unused. Your code decides afterwards what matters for the case at hand. S74 For an online shop that sorts hundreds of messages a day, this means less waiting on the network and simpler code. The pattern does, however, have limits that the page does not mention.

What the documentation example looks like

The example triages customer support tickets. One request contains the ticket text and five questions: Choice category (the code uses the values bug_report, billing and feature_request), Score bug_severity, Noul has_reproducible_steps, Noul refund_requested and Score frustration. Bug severity and described reproduction steps only matter for bug reports, the refund question only for billing, and the code checks frustration on every ticket. S74

The code then branches. A bug report with a severity score above 1.5 and a probability of described steps above 0.6 goes to the engineers; other reports end up in the bug list. A billing ticket with a refund-request probability above 0.7 gets a flag, a feature request is simply logged, and frustration above 1.5 means a priority reply regardless of category. S74

The documentation contrasts the pattern with an approach in which the category is determined first and severity is added with another call. When a speculative answer is needed, the second call is no longer necessary; when it is not, the code simply leaves it unused. S74

What the pattern promises and what it leaves out

According to the documentation, the questions are evaluated in parallel, so adding more usually does not lengthen the response time much. S74 This is a vendor claim; the page gives no measured values for it. S74 The API description adds that all questions in a request refer to one shared state and that question types can be mixed in a single request. The API description does not state a maximum number of questions per request. S75

Caution is needed on cost. TypeSafe bills input tokens, at 0.042 USD per million for both Jev 1.14 and 1.13, and output tokens are currently free. S01 The response reports usage in one usage object for the whole request, not per question. S75 Neither the pattern page nor the API description explains how many input tokens each additional question adds to a request. S74 S75 The logic of the pattern implies that you send the shared text once instead of in several separate calls, but check the actual savings against the input_tokens value on your own data. The cost calculator can help you estimate costs.

Size limits also apply. By default they are 64k tokens per request and 32k per question; with the header TypeSafe-Danger-Zone-Long-Context: true, both limits rise to 262k. S01 More details are in the pricing and limits guide.

The thresholds in the example are not recommendations

The values 1.5, 0.6 and 0.7 in the example are for illustration only. For a Score question, the result is a weighted average of level numbers, which are numbered from zero, so with three levels the score runs from 0 to 2. S04 The fan-out page does not say how many levels bug_severity and frustration have. S74 You therefore cannot read the meaning of the 1.5 threshold from the example: with three levels it sits near the top of the scale, with five (0 to 4) below its middle. This is our own calculation, not a vendor figure.

For Noul questions, the documentation suggests starting with a threshold of 0.7 for yes and 0.3 for no and then adjusting the values to your own results. S06 How to set and validate thresholds is covered in the confidence and calibration guide.

The example also lacks a branch for a ticket that falls into none of the three categories in the code. S74 In production, we recommend adding such a branch and sending these cases to a human.

The pitfall: one shared state for all questions

Fan-out tempts you to put everything any question needs into a single state. Yet the page on the weaknesses of Jev 1.13, last reviewed on 2 October 2026, warns that accuracy drops as the state grows with content unrelated to the decision. It recommends selecting in code beforehand only the fields the question needs. S07 The page concerns version 1.13; we did not find a separate overview of weaknesses for 1.14 in the documentation.

Our suggestion: put questions that work with the same input into one request, for example the text of a single message. When a question needs different data, such as order history, send it separately with its own, shorter state.

What this means for a Slovak online shop

Take the message "Balík neprišiel, chcem peniaze späť." ("The parcel didn't arrive, I want my money back.") Following the pattern, one request can contain a Choice question on the message's intent, a Noul question on the refund request and a Score question on the level of dissatisfaction. The code builds the next steps from the answers: the intent picks the queue, the refund probability decides on a flag for the accountant, and high dissatisfaction moves the message forward. This split of work between model and code matches the documentation example. S74 We described intent-based routing of messages in more detail in the article on sorting customer messages.

Two things, however, you should check yourself. The pattern's documentation says nothing about Slovak, so set thresholds only after testing a sample of your own messages; the testing in Slovak guide describes the procedure. And because the code combines the result from several separate answers, be prepared for cases where they contradict each other. How answers to related questions can diverge is discussed in our article on consistency between Choice and Noul.