Sorting customer messages with Jev: the intent routing pattern

In its documentation, TypeSafe describes a pattern called intent routing. It rests on the idea that different kinds of requests are worth handling in different ways. Jev sits at the entry point as a fast, cheap classifier, and behind it wait fixed code, a specialised language model or a human. S58 For an online shop that receives questions about parcel status, return requests and complaints every day, this is one of the most direct ways to put a decision model to work. This article walks through the pattern based on the vendor's documentation; JEV.sk did not measure it.

Two questions in one request

The example in the documentation comes from customer support. A single request to TypeSafe contains two questions at once. S58 The first is a Choice question and determines the main intent of the message from four options: order_status for a question about an existing order, product_question for a question about a product before buying, return_exchange for a return or exchange, and complaint for an unhappy customer who wants a resolution. S58 The second is a Score question and rates complexity on a scale from 0 to 2. Zero means a simple lookup or standard procedure, one requires judgement or several steps, and two marks an unusual case that may need escalation. S58

There is a practical reason to ask both questions in one request. The documentation for the Choice question advises sending all the questions your code might need at once, because they are evaluated in parallel. Latency therefore grows only a little, although tokens are added. S03 The difference between question types is explained in the guide Choice, Score and Noul.

Code decides the path, not the model

Jev returns only answers. Where the message goes next is decided by your application, following rules that the documentation lists in this order: S58

1. When the confidence (confidence) for the intent is below 0.5, the message goes to a human.
2. order_status is handled by fixed code that looks up the order without a language model.
3. product_question goes to a language model focused on products.
4. return_exchange goes to a model focused on returns.
5. complaint goes to a human if complexity is above 1 or the confidence for complexity is below 0.5. Otherwise it is handled by a model intended for resolving complaints.

According to the documentation, more expensive resources are thus used only when a request actually needs them. S58 On the additional confidence check for complexity, the page adds that what low confidence means should be judged against the system's context and what is at stake in the decision. S58

What the 0.5 threshold actually means

Confidence is a number from 0 to 1 derived from the distribution of probabilities. A value of 1 means all the probability went to one option, a value of 0 means an even spread. S06 For a Choice question with n options, the documentation gives the formula (p_max − 1/n) / (1 − 1/n), where p_max is the highest probability. S06 If we plug in the four intents from the example, a confidence of 0.5 corresponds to a highest probability of 0.625. This is our own calculation from the formula, not a figure from the vendor. A threshold of 0.5 therefore does not mean the model is split fifty-fifty between two options.

The vendor does not consider specific threshold values universal. In its view, the right limits depend on the domain and on how the model performs for the given use, so it recommends starting conservatively and checking them on your own data. S06 The page also does not claim that confidence matches actual accuracy. S06 This is covered in more detail in the guide confidence and calibration.

What a Slovak online shop can take from this

The four categories from the example carry over to the support of a typical Slovak online shop almost unchanged. The following points are a suggestion from JEV.sk, not part of the documentation.

• Leave order status to code. The pattern sends order_status straight to a lookup without a language model. S58 When a message contains an order number, the reply is assembled by the database and a template, and no generative model is needed.
• Write option descriptions that set the options apart. The model does not see the question identifier, but it does see the option names and their descriptions. S03 Options that are easily confused can be described with an object instead of a short string, with fields such as what the option covers, what it is not for, and example inputs. S03 For us this would fit, for instance, when telling a product return apart from a claim for a defect.
• Plan for messages outside the list. When the list may not cover every input, the documentation recommends adding an other or "none of the above" option. At the same time it admits that this option may not work well in many cases, and as an alternative it offers a separate Noul question that checks whether any of the options fits. S03 Choice accepts up to 255 options. S03
• The model must not close a case on its own. The pattern leaves complex complaints to a human. S58 As we also write in the guide Jev for online shops, the classification result is a routing suggestion, not a decision on the customer's claim.

Classification is cheap, but not free. TypeSafe charges 0.042 USD per million input tokens, and output is free of charge. S01 Each additional option adds a few tokens, and so does each additional question. S03 A related idea, Jev as a dispatcher in front of more expensive agents, is also the subject of our article Jev as a dispatcher for multiple agents.

Where the pattern falls short

The documentation shows a skeleton, not a finished system. It does not state how accurately Jev determines intent in Slovak, nor does it name the specific model version used in the example. S58 Before deployment, we therefore recommend going through a sample of real messages, comparing the result with manual classification and only then setting the thresholds. How to approach Slovak texts is described in the guide testing in Slovak.