We replaced our decision stack with Jev. It worked.

Ayush · · 5 min read

Someone says "okay" during checkout. Do we move to the next instruction, or do we need to answer a question? We used to ask a generative model to work that out. It could, but a simple acknowledgement shouldn't require a fresh piece of open-ended reasoning.

Elsewhere in the call, we used embedding similarity and KNN to make four other decisions. Each classifier was reasonable on its own. Together they shared one local model and competed for its attention.

We replaced these paths with Jev, TypeSafe's typed decision model. Jev doesn't write replies. It answers bounded questions with choices and probabilities, and our code decides what to do with them. It now runs on live calls. The biggest lesson wasn't that a new model could answer our questions. It was that we had to get much clearer about the questions and who had authority to act on the answers.

Where we used to make decisions

Consent, payment responses, routing and liveness were four independent embedding + KNN decisions. They weren't stages of a pipeline. They just happened to rely on the same local model, with a shared lock that could serialize work across a voice call.

Checkout had a different shape. Our conversational LLM was already there, so we let it interpret whether a caller had acknowledged an instruction or reported progress. That made sense for questions and problems, but it was too much machinery for routine transitions. Worse, interpreting speech and deciding whether a payment has happened are not the same job.

Before · Embedding + KNNindependent decisions
ConsentPaymentRoutingLiveness
Shared model lock
Now · Jevtogether
One batched Jev request
ConsentPaymentRoutingLiveness
Four independent embedding + KNN decisions once shared a local model lock. We now ask Jev all four questions in one request.

One request, four questions

The call orchestration layer now asks Jev whether a response means yes or no to consent, yes or no to payment, which route fits the turn, and whether an utterance is a liveness probe. The questions are independent, so we send them together instead of making four separate trips.

A batched request would still be too slow if it began only after speech recognition finished. We start it when silence begins, while the call finishes detecting the turn and finalizing the transcript. If the answer isn't ready when needed, the ordinary path takes over. We don't make a caller wait for a speculative decision.

Silence beginsDecision needed
Speech finalizes
Jev decision

The call never waits for a late answer.

Jev starts at silence onset, in parallel with speech finalization. If its answer is late, the ordinary path continues.

Early production traffic shows that when a prefetched result is available to consume, it is usually ready in time. That's useful evidence that the overlap works; it isn't a claim that every call is faster. More important, these four decisions no longer queue behind the local embedding model.

Checkout without a needless LLM turn

Checkout needed a separate integration. A caller can acknowledge an instruction, report seeing a payment screen, ask for help, describe a problem or change their mind. Jev classifies the response and, when someone reports progress, identifies the milestone they described.

The billing system supplies the milestones that are actually possible at that point. A clear acknowledgement can follow a deterministic path without asking the LLM to speak again. A progress report can do the same only when Jev identifies an allowed milestone confidently enough. Questions, problems and uncertain answers still go to the LLM. And Jev never declares a payment complete: only verified payment events and billing state can do that.

One checkout turn · Two possible paths

Deterministic fast path

Clear acknowledgement

or an allowed progress milestone

The billing system checks what can actually advance.

Full language path

Question or uncertainty

Questions, uncertainty and everything that needs a real reply stay with the LLM.

A clear acknowledgement or allowed milestone can skip generation. Questions and uncertainty keep the conversational path; verified billing state decides whether payment is complete.

We are seeing that path used on live checkout turns. It is a small part of checkout, as it should be: most turns still need a real reply or don't meet the fast-path conditions. The early win is not that Jev replaces conversation. It is that an ordinary acknowledgement doesn't have to become another conversational-model turn, while payment authority stays with the billing system.

Ask about the caller's world, not ours

Our first checkout question described progress in our own internal labels. Callers don't speak in those labels. Someone might say "bank ka page aa gaya"; our question had to recognize the screen they meant, not the name we gave it in code. In a small hand-labelled replay, changing the criteria to describe what callers actually say turned vague progress answers into concrete observations. We didn't fix that by lowering the confidence threshold. We fixed the question.

That lesson applies beyond checkout: describe the decision in the language of the person you're listening to, not your internal enum names.

Confidence is application policy

We didn't give every Jev answer the same threshold or the same fallback. Consent and payment are asymmetric: mistaking a "no" for a "yes" is worse than asking again. When either answer is uncertain, we keep it ambiguous rather than advancing. Routing can return to the ordinary conversational handler. Liveness can retain its existing verdict.

That separation matters more than a single accuracy figure. The model offers a judgement; the application owns the consequence. In checkout, that means Jev may notice what a caller says they see, but it never gets to verify money moving.

What we want to try next

Jev already handles bounded decisions around a call. Next, we want it to give the conversational model hints about which tool capabilities fit what the caller is trying to do. The LLM would still choose the specific tool and speak to the caller. Jev's job would be to narrow the choices, not to take over the conversation.

The goal is fewer wrong tool choices and fewer made-up answers, without making callers wait longer or blocking a tool they actually need. That is a direction we're evaluating, not a benefit we've shipped or measured.

The principle stays the same: let Jev answer a narrow question, and keep the authority to act in the system that owns it.

References and further reading